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Chapter One 



INTRODUCTION 



PURPOSE OF THE PROJECT 

In the post'-Sputnik era, "language for communication" emerged as the 
major goal of foreign language programs in the United States. Techniques and 
materials for developing oral skills began to replace more traditional 
approaches to modern Ismguage teaching in classrooms at all levels. To 
achieve the goal of communication, however, foreign language educators 
realized that evaluation plays as important a role in the teaching/learning 
process as curriculum development and classroom instructional methods. They 
recognized the critical need to develop procedures and instruments to evaluate 
foreign language proficiency in the four skills and in the speaking skill in 
particular. 

PROJECT OPT was a research and training project in oral proficiency 
testing in college-level foreign language courses conducted at the University 
of New Hampshire under a grant from the U.S. Department of Education in 1982- 
84. It was designed to achieve three objectives: 

1. to determine the feasibility of incorporating specific techniques of 
oral proficiency evaluation in foreign language courses at the college level; 

2. to create a network of foreign language professors in northern New 
England colleges and universities trained in the use of oral proficiency 
evaluation procedures; and 

3. to develop a handbook of oral proficiency evaluation techniques for 
college-level foreign language faculty. 

RATIONALE £QR !&£ IBQJESI 

A concerted effort to develop valid and reliable techniques for assessing 
oral proficiency in academic settings began in 1979. The publication of the 
report of the President's Commission on Foreign Language and International 
Studies, Strength Through Wia^gm (1979), served as a catalyst to the academic 
community. In response to the Commission's recommendation that langueige 
proficiency goals and guidelines be established, the American Council on the 
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Teaching of Foreign Languages (ACTFL) auid the Educational Testing Service 
(ETS) Initiated several projects related to oral proficiency testing. The 
primary result of these efforts was the ACTFL/ETS oral proficiency Interview, 
a procedure based on the original Foreign Service Institute model but adapted 
to meet the needs of academic settings. 

The ACTFL/ETS projects focused on the development of procedures and the 
training of college faculty In use of the techniques of Interviewing and 
rating. Both the developers, who of necessity were the first trainers, and 
the trainees accumulated considerable knowledge and experience through use of 
the techniques In workshops and subsequent applications on home campuses. A 
comprehensive Implementation of these techniques seemed a reasonable and 
needed next step In the development process. PROJECT OPT was designed to 
fulfill that need. 

The three objectives of PROJECT OPT provided the framework for examining 
the utility and effectiveness of the ACTFL/ETS Oral Proficiency Interview in 
the context of college-level foreign language courses. This examination 
focused on factors relevant to the outcomes of training and the implementation 
of specific techniques for assessing oral proficiency in academic settings. 
ORGANIZATION OF THE REPORT 

This report of the project discusses how the three objectives were 
realized. Each objective is considered as a separate, albeit related, 
component of the project and is treated in the chronological order in which it 
occurred. The three components are: (1) the training and networking 
component, (2) the research component, and (3) the materials development and 
dissemination component. Appendices containing documents used in the 
project complete this report. 
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Chapter Two 
THE TRAINING AND NETWORKING COMPONENT 

The first phase of PROJECT OPT consisted of a training component in which 
foreign language faculty from colleges and universities in the northern New 
England area received instruction in the use of the ACTFL/ETS Oral Proficiency 
Interview. This section of the report describes several aspects of the 
training component. Also discussed are the networking activities that 
resulted from participation in the project. 
OVERVIEW OF THE TRAINING COMPONENT 

The training component of the project, which was modeled on the first 
ACTFL oral proficiency training workshop held in Houston in February of 1982, 
consisted of an intensive four-day workshop, a practicum on the home campus, 
and a two-day follow-up workshop. The Directors of the project worked closely 
with the Workshop Coordinator, Judith Liskin-Gasparro of the Educational 
Testing Service and with C. Edward Scebold, Executive Director of the American 
Council on the Teaching of Foreign Languages, to develop an agenda that would 
provide the necessary training and fulfill the requirements of the research 
component of the project. 
SELECTION OF PARTICIPANTS 

Participation in the project was limited to professors engaged in 
teaching college-level foreign language courses. The college-university level 
was selected because of three factors: 

1. Interest on the home campus of the principal investigators. 

2. Previous successful training workshops for post- secondary level 
professors of foreign languages. 

3. The potential for influence on other levels associated with college- 
level foreign language teaching. 

Languages included in the project were initially the most commonly taught 
in post- secondary institutions in the United States and in northern New 
England: French, German, and Spanish. At the request of the Dean of the 
Middlebury College Russian School and others, Russian was added to the list of 



languages, thus providing the first training opportunities for professors of 
that language. 

Faculty rosters, obtained from department chairpersons, constituted the 
list of 240 possible participants who received an announcement of the project. 
The original design called for thirty participants with ten persons in each of 
the three languages of French, German, and Spanish. Because of the nature of 
the applicant pool and the requests for training in Russian, the final group 
consisted of eight French, four German, four Russian, and nine Spanish 
professors. These 25 workshop p£u?ticipants represented 14 colleges and 
universities in the states of Maine, Massachusetts, New Hampshire, New York, 
and Vermont. 
IHE TRAINING WORKSHOP 

The four-day intensive training workshop was held on January 3 through 7, 
1983 at the University of New Hampshire in Durham, N.H. The agenda for the 
workshop included intensive training in the design, administration, and 
scoring of oral proficiency tests using the ACTFL/ETS oral proficiency 
interview and rating procedures. Judith Liskin-Gasparro, Senior Evaluator for 
the Educational Testing Service and a primary developer of the ACTFL/ETS oral 
proficiency interview, served as Workshop Coordinator and Consultant in 
Spanish. Dr. Pardee Lowe, Chief of Testing for the Central Intelligence 
Agency Language School and the principal designer of oral proficiency 
evaluation techniques for the U.S. government, was Consultant in German and 
Russian. Dr. Alice Omaggio, Assistant Professor of French at the University 
of Illinois (Champagne-Urbana) and an author of the ACTFL Proflciencv 
Guidelines, was Consultant in French. Each participant received a copy of the 
ETS Oral Proflciencv Testing Manual, which was used as the primary text for 
the entire project. 
TRAINING AND PROJECT INTERVIEWS 

Consistent with the training programs designed by ACTFL and ETS for oral 
proficiency interviewers, participants in PROJECT OPT returned to their home 
campuses after the four-day intensive workshop and conducted two sets of 
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interviews between the period of January 8 and May 15, 1983 . 

Training interviews . Participants conducted and rated eight interviews 
on the home campus using students from all levels of proficiency. Tapes of 
three randomly- selected interviews of each participant were critiqued by the 
three workshop consultants and Mrs. Maria Casey of the CIA Language School who 
critiqued the tapes of the four Russian participants. Following receipt of 
the written critique, each participant was contacted by telephone by the 
appropriate consultant. The purpose of this conversation was to provide an 
opportunity to receive further clarification and suggestions for conducting 
the second round of interviews. 

Proiect interviews^ After the written and telephone critiques, 
participants conducted 16 more oral interviews in two designated grot ^ Tb6 
eight interviews in Group A were conducted using students who were cuxi^t^oxy 
studying with the interviewer. The eight interviews in Group B were conducted 
using students who had not studied, nor were currently studying, with the 
interviewer. This procedure was used to collect data for the research 
component of the project and will be discussed more fully in the next chapter 
of this report. 

To provide the necessary objectivity for unbiased evaluation, the project 
interviews were critiqued by a second set of consultants who were experienced 
testers and raters. These consultants were: 

French; Dr. Carol Murphy, University of Florida 
German; Dr. Heidi Byrnes, Georgetown University 
Russian; Dr. Ellen Mitchell, Defense Language Institute 
Spanish; Dr. Frank Medley, University of South Carolina 
Four randomly-selected tapes, consisting of two tapes from Group A (students 
known by the participant) and two from Group B (students not known by the 
participant) were critiqued using evaluation procedures identical to those 
employed in the practice interview phase. Report procedures were different 
and are described below. 



FOLLOW-UP WORKSHOP 

A follow-up workshop was held at the University of New Hampshire on 
August 5-7 1 1983. At that time, participants met with the Directors, the 
Workshop Coordinator, and the second set of consultants who had critiqued the 
project tapes. In the absence of Dr. Heidi Byrnes, Consultant in German, Dr. 
Ilse Christoph of the CIA Language School, served as consultant in German. 
During the two-day workshop, further training in interviewing, situation 
development and rating was provided. A sample interview in each language was 
videotaped and group discussions on implications for curriculum and materials 
development were held. 
NETWORKING ACTIVITIES 

Under the leadership of the directors of PROJECT OPT, the participants in 
the project formed the nucleus of a network of professors interested in oral 
proficiency testing in northern New England. During the second year of the 
project, a newsletter entitled POPT (Publication of Project OPT), served as a 
vehicle of communication among participants. (See Chapter Four.) 

Several oral proficiency workshops were conducted by the directors and 
participants, both in the geographical region of the project and in other 
parts of the country to stimulate interest in the network. (i>ee Chapter 
Five.) Although funding for the project has ceased, the network is continuing 
to function and have an impact on foreign language teaching, expecially in the 
state of New Hampshire. This impact is discussed in Chapter Five of this 
report. 
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Chapter Three 
THE RESEARCH COMPONENT 

The research component constituted the principal objective of PROJECT 
OPT: to determine the feasibility of incorporating specific techniques of oral 
proficiency evaljiation in foreign language courses at the college level. This 
chapter describes the research questions, population and sample, plan of 
action, instrumentation, data collection and analysis procedures, and results 
of the study that was conducted at the University of New Hampshire in 1982-84. 
RESEARCH QUESTIONS 

The study was a descriptive one in which the primary purpose vac to 
gather and analyze data relative to the feasibility of incorporating specific 
techniques of oral proficiency evaluation in foreign language classes at the 
college level. Nine research questions formed the basis for investigation; 

1. Can professors learn to conduct and rate oral proficiency interviews 
effectively after completing an intensive four-day training 
workshop? 

2. What demands, in terms of time needed for preparation, 
administration, and scoring of the oral proficiency test, does use 
of the oral proficiency interview make upon professors? 

3* What special conditions, facilities, and equipment do professors 
need to use the oral proficiency interview? 

4. What impact does the oral proficiency interview have upon the 
curriculum, the instructional materials, and the procedures used to 
achieve the goals of the course? 

5. What presage and context variables are associated with successful 
use of the oral proficiency interview? 

6. What problems do professors encounter when using the oral 
proficiency interview in a foreign language course? 

7. What further training is necessary to help professors test oral 
proficiency in their courses? 

8. What differences, if any, appear in the interviewing and rating 



techniques of professors when interviewing their own students and 
students whom they do not know? 
9. What proficiency levels are associated with typical levels of 
instruction (beginnlngi Intermediatei advanced) at the college 
level? 

These questions addressed four major areas of concern: 

1. Efficacy of training in oral proficiency techniques: Question 1. 

2. Factors related to use of the oral proficiency interview in college- 
level foreign language courses: Questions 2 through 7. 

3. Factors related to rating one*s own students: Question 8. 

4. Relationships between proficiency levels and instructional levels: 
Question 9. 

POPULATION AND SAMPLE 

The population for this study consisted of all French, German, Russian, 
and Spanish professors who were teaching in a college or university in the 
northern New England states of Maine, New Hampshire, and Vermont during the 
1982-83 academic year. Faculty rosters, obtained from department 
chairpersons, were used to personally invite the 240 eligible professors to 
participate. 

The original design called for a sample of thirty participants with ten 
professors in each of the three languages of French, German, and Spanish. 
Because of (1) the nature of the applicant pool and (2) requests by several 
Russian professors, including the Director of the Middlebury College Russian 
Summer School, the final sample, which was self-selected, consisted of eight 
French, four German, four Russian, and nine Spanish professors. They came 
from public (14) and private (11) colleges (17) and universities (8) in the 
state of Maine (4), Massachusetts (2), New Hampshire (11), New York (2), and 
Vermont (6). One participant in French withdrew from the project in the 
training phase and the data from one Spanish participant did not arrive in 
time for analysis. 
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PLAN OF ACTION 

The study consisted of six phases as follows: 

Phase 1: October 1982 to January 1983: Design of research procedures 
and Instruments. Included development of self-report questionnaire for data 
collection; planning of training workshop; promotion, publicity and recruit- 
ment of participants; coordination of workshop activities, meals, and lodging. 
Accomplished by the directors of the project in consultation with the Workshop 
Coordinator, Judith Liskin-Gasparro of the Educational Testing Service. 

Phase 2t January 3-7, 1983: 

(A) Participant training workshop. Included intensive training in 
the design, administration, and scoring of oral proficiency tests using the 
ACTFL/ETS oral interview and rating procedures. (See Chapter Two.) 

(B) Collection of data relative to variables associated with presage 
and context variables. (See Instrumentation section below.) Preparation for 
participant project in oral proficxency interviewing and rating. 

Phase ^: Jan\j|ry 8 - May 15, 1983: 

(A) Participant project implementation on home campuses. 
Participants completed a project using the techniques learned at the training 
workshop. This project consisted of conducting and rating a set of eight 
training interviews and a second set of 16 prolect interviews. 

1. Training interviews: 8 interviews were conducted and 
rated by participants using students from all levels of 
proficiency. Three randomly-selected tapes of each 
participant were critiqued by the language consultants 
from the training workshop. These consultants sent a 
written report to the participants and subsequently spoke 
by telephone with the participants to provide further 
clarification and suggestions for conducting the second 
set of interviews. 

2. Project interviews: After the written and telephone 
critiques, participants conducted 16 project interviews in 



two designated groups* The eight interviews in Group A 
were conducted using students who were currently studying 
with the participant. The eight interview candidates in 
Group B were drawn from a pool of students who had not 
studied nor were currently studying with the participants. 
(B) Data collection: Upon completion of the 16 project interva.ews, 
participants supplied data relative to their experiences in using the 
procedures (See Instrumentation section below). 
Phase 4: May 15 - August 4, 1983 

(A) Critique of participant project tapes by a second set of 
consultants. Four randomly-selected tapes, consisting of two tapes from Group 
A (students studying with the participant and two from Group B (students not 
known to participant) were critiqued by the following consultants who were 
trained evaluators and raters: 

French: Dr. Carol Murphy, University of Florida 
German: Dr. Heidi Burns, Georgetown University 
Russian: Dr. Ellen Mitchell, Defense Language Institute 
Spanish: Dr. Frank Medley, University of South Carolina 

(B) Data collection: The consultants (hereafter called 
"evaluators") provided data relative to: (1) their perception of the 
relationship of interviewer and candidate and (2) their evaluation of the 
participants* rating and evaluation techniques. 

(C) Development of plans for follow-up workshop held in August, 
lhaae 5.: August 5-7, 1983: Participant Follow-Up Workshop: Participants 

returned to the campus of the University of New Hampshire to meet with the 
Directors, the Workshop Coordinator, and the Evaluators to discuss projects, 
problems, and future needs. 

Phase ^: Completion of project: Final preparation of the Handbook, 
completion of the data analysis, and preparation of the Final Report of the 
project. 
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INSTRUMENTATION 

Three instruments were designed to collect data for the study; 

1. Questionnaire I: Participant Questionnaire 

2. Questionnaire II: Participant Project Report 

3. Questionnaire III: Consultant Report of Interviewer Elicitation and 
Rating Techniques 

In addition, several forms were developed to collect additional relevant data 
Examples of the three questionnaires and the forms used in the study are found 
in Appendix B. 

Questionnaire Xl Participant Questionnaire 

This instrument, which was administered to all participants on the second 
day of the training workshop, provided data pertaining to teacher presage 
variables (teacher characteristics and training experiences) and to context 
variables (course and program objectives, instructional techniques and 
materials). The four sections of the questionnaire focused on (1) 
preparation, (2) professional development, (3) curriculum and instruction, and 
(4) background information. (See Appendix B for a copy of this 
questionnaire) . 

The primary function of this questionnaire was to supply data about the 
characteristics of the sample used in the study. The two intended purposes 
were: to determine (1) what kinds of professors are likely to seek training in 
oral proficiency evaluation and (2) what relationships, if any, exist between 
the presage and content variables investigated in the questionnaire and 
success in training. These purposes relate to research question 1. 
Questionnaire II; Participant Pro.iect Report 

This instrument, which was completed by participants upon finishing the 
set of 16 project tapes, provided data pertaining to experiences and attitudes 
related to use of the oral proficiency evaluation techniques. The six 
principal sections of the questionnaire dealt with (1) time factors, (2) 
facilities and equipment used, (3) the specific course from which Group A 
interview candidates (students of the interviewer) were drawn, (4) experiences 




with interview candidates in Group A (students studying with the participant) 
and Group B (students unknown to the participant), (5) problems encountered in 
interviewing and rating and (6) need for further interviewer training and 
development of oral proficiency evaluation techniques. (See Appendix B for a 
copy of this questionnaire. ) 

The primary function of this questionnaire was to supply data in order to 
answer research questions 2, 3, 4, 5, 6, and 7. 

Questionnaire III: Consultant Report of Interviewer Elicitation and Rating 
Techniques 

This instrument, which was completed by the four language-specific 
evaluators who critiqued the project interviews, provided data relative to 
participant techniques. The three sections focused on (1) elicitation 
techniques, (2) rating known and unknown students, (3) a global rating of 
elicitation and rating techniques. (See Appendix B for a copy of this 
questionnaire . ) 

The primary function of this questionnaire was to provide answers to 
research questions 1, 5, and 8 of the study. 
Miscellaneous instruments 

In addition to the three questionnaires, several forms were developed to 
report relevant data at various stages of the project. These included: 

1 . Participant Rating Sheet for Practice Interviews 

2. Participant Rating Sheet for Project Interviews 

3. Participant Project Log 

4. Evaluator Rating Sheet 

Examples of these instruments are found in Appendix B. 
DATA COLLECTION AND ANALYSIS 

Data for the study were collected, using the previously-described 
instruments, according to the following schedule: 

1. Questionnaire I, Participant Profile: 5 January 1983 

2. Questionnaire II, Participant Project Report: 15 May, 1983 
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3. Questionnaire III, Consultant Report of Interviewer Ellcltatlon and 

Rating Techniques, 15 July 1983 
Data analysis was accomplished In the following manner for each of the 
questionnaires: 

1. Questionnaire I: Responses were totaled and reported In terms of 
frequencies and, when appropriate. In percentages. (See Appendix 
C.) 

2. Questionnaire II: Given the open-ended nature of the items, 
responses were analyzed and subsumed into principal categories. 
These were reported in frequencies and, when appropriate, in 
percentages. (See Appendix C./ 

3. Questionnaire III: Responses were totaled and reported in terms of 
frequencies and percentages. (See Appendix C.) 

RESULTS OE THE STUDY 

The results of the data analysis are reported below in two sections 
related to the characteristics of the sample and the research questions 
investigated in the study. 
Characteristics of the sample 

'The data collected in Questionnaire I, Participant Questionnaire, 
pertained to teacher presage variables (teacher characteristics arid teacher 
training experiences) and to context variables (course objectives. 
Instructional techniques and materials). Because of the self-selective nature 
of the sampling process, the data provide information on the kind of professor 
who is likely to want to become involved in oral proficiency testing. 

In addition to this descriptive function, the data was originally 
Intended to be used to determine possible relationships between presage and 
context variables, on the one hand, and demonstrated success in using the oral 
proficiency test, on the other hand. This analysis was not realized because 
of two factors: (1) the small size of the sample in the language-specific 
groups (French n = 7, German n = 4, Russian n = 4, and Spanish n = 9) and (2) 

ER?C '3 



the impossibility of determining interrater reliability coefficients across 
the four language groups. 

The raw data from the questionnaire, reported in frequencies are found in 
Appendix C. An analysis of these data follows. 

Presage ILarlables^ Characteristics, The typical participant in this 
study was likely to be male (545f), over 38 years of age (5451), a native 
speaker of English (7155), a recipient of the Ph.D. (585f) in literature (675f), 
non-tenured (SSjf), and a veteran of at least four years of teaching at the 
college level (66^). Implications of these data include the following: 

1 . Despite the fact that tenure-track positions in foreign language 
departments are predominantly held by men, women are almost equally 
represented in this study. 

2. Older faculty members are interested in learning about and 
implementing new techniques of language pedagogy. 

3. While non-tenured faculty members, who often bear the responsibility 
for teaching and coordinating language courses, outnumber tenured 
professors, the relatively high participation of the latter 
indicates interest on the part of the continuing faculty, who hold 
the real power in departments, in improving instruction* 

4. Professors who received their doctorates in literature recognize the 
need to keep abreast of new developments in the teaching and 
learning of foreign languages. 

5. Experienced faculty members seek professional enrichment in the area 
of foreign language pedagogy. 

Presage Variables: Teacher Training experiences . As undergraduates, 
slightly more than a third of the participants either received academic credit 
for a study-abroad program in the target country {38%) or lived or travelled 
for more than three months in such a country (335f). Similarly, slightly more 
than a third of the participants took a formal course in foreign language 
methodology (385f) while (425f) student-taught a foreign language. 

At the graduate level, less than one third of the participants received 
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academic credit for study abroad {29%) but more than half lived or travelled 
for more than three months in a -target country. At this level, slightly more 
than a third had taken a formal course in foreign language methodology (36%) 
but almost three-quarters of the participants taught a language course (71^) • 

Based on these data and allowing for overlap between undergraduate and 
graduate experiences, it is likely that slightly more than half of the 
participants had received credit for study in a target country and/or lived or 
travelled for more than three months in a target country. The implication is 
that these participants had had the opportunity to develop their own oral 
proficiency and cultural awareness while in the target country as part of 
their professional training experiences. In addition, it is likely that half 
of the participants had taken a formal course in foreign language methodology 
while at least three-quarters had taught language courses before completing 
their degree. 

Presage Variables: Professional Growt h, Several questions related to 
professional growth and involvement can be interpreted as an index of interest 
in and awareness of new developments in the field. At least three quarters of 
the participants had spent a week or more in a target country in the past 
three years while virtually all (965f) of the participants were members of a 
foreign lamguage professional organization. Two thirds usually or always 
attend a session on foreign language methodology at professional meetings and 
almost half (46^) regularly read professional Journals of foreign language 
teaching. One third reported that they had previously attended some type of 
oral profiency evaluation workshop. Thus, the participants manifested 
interest in maintaining their own oral skills as well as keeping up with 
developments in the profession. 

Context variables. Context variables refer to aspects of curriculum and 
instruction that affect learning and teaching. Data collected in this study 
provided a description of the kinds of objectives, teaching methodologies, and 
evaluation procedures used in the various departments represented by the 



participants. Three levels of Instruction (elementaryi Intermedlatei and 
advanced) were compared. 

At the elementary and Intermediate levels, three quarters of the 
participants reported use of stated departmental oral objectives for courses, 
while at the advanced level only 29% reported having specific oral objectives. 
Half of the participants reported that the student's speaking ability 
constitutes at least 20% of the grade at the elementary level while only one 
quarter Indicated the same percentage at the Intermediate level and slightly 
less (2^%) at the advanced level. The target language was always or almost 
always the language of instruction in two thirds of the courses at the 
elementary and intermediate levels (6355) and in almost three quarters {7^%) at 
the advanced level. At the elementary level, 83% of the participants reported 
giving speaking tests at least once a semester. This percentage declined to 
63% at the intermediate level and k2% at the advanced level. Oral proficiency 
tests for placement and/or some type of validation, such as fulfilling a 
requirement for language study, graduation or teacher certification, were 
reported by one third of the participants. 

Analysis of the data indicates that interest in stating oral objectives 
and formally evaluating oral performance is highest at the elementary level 
and lowest at the advanced level. This is interesting to note, given the 
usual practice of offering courses of conversation and composition at the 
advanced level of language instruction. It would appear that less formal 
definition and evaluation is used at the advanced level even though classtime 
Is reported to be spent in developing oral skills. Professors may believe 
that formal objectives and evaluation are not necessary since they are in 
dally contact with students and can assess their speaking ability without 
using specific instruments or techniques. 

Syipmary characteristics ih& sample. The typical participant was an 
experienced, non-tenured professor with the doctorate in literature who had 
studied or traveled in the target country with]^the last three years. This 
person belonged to a professional foreign language association, attended 
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sessions on methodology at professional meetings, and used speaking objectives 
and some form of speaking tests in his or her courses. While this profile is 
probably not typical of the majority of foreign language professors in U.S. 
colleges and universities, it is likely that at least one such person exists 
in most departments. The extent to which that person can convince colleagues 
of the need for and efficacy of oral proficiency evaluation is a critical 
issue. 

Research Questions 

The data for addressing the nine research questions of the study were 
obtained from Questionnaire II, the Participant Project Report, and 
Questionnaire III, the Consultant Report of Interviewer Elicitation and Rating 
Techniques. A complete summary of the data is found in Appendix C. Answers 
to the research questions, based on selected data presented in Tables 1 
through 11 are discussed below. 

•Question 1 . Csa professors igacn ia conduct and rate oral proficiency 
interviews effectively after completing an Intensive four-dav workshop and a 
supervised practicum that inoludea ^ critique ^ 2i interviews ?: 

The answer to this question is a qualified yes. When the study was 
conducted, a procedure for certifying new testers had not yet been approved by 
the Interagency Language Roundtable. Thus it was not possible, at that time, 
to state definitively if the type and length of training was adequate to lead 
to certification of a high percentage of the workshop participants. However, 
the evaluators who critiqued the tapes did judge several criteria related to 
the participants* performance and potential. Table 1 (see next page) 
summarizes the data for all participants across the four languages used in the 
study. As Table 1 indicates, the evaluators gave high ratings to 57% of the 
participants for interviewing techniques, to 30% for rating techniques, to 70% 
for potential as a proficient interviewer, and to 65% for potential as a 
proficient rater. A uniform ^3% received low ratings in all categories, with 
the remaining percentages falling in the average category. 
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Table 1 

Evaluator Critique of Participant Techniques and Potential 

ALL LANGUAGES 



Criteria 


High 


Average 
n/% 


Low 
n/% 


Total 
n/% 


Interviewing 
techniques 


M/51% 


1/30% 


3/13JI 


23/100? 


Accuracy of 
rating 


1/30% 


13/57JI 


3/13JI 


23/100? 


Potential : 
proficient 
interviewer 


l6/705( 


4/17? 


3/13? 


23/100? 


Potential : 
proficient rater 


15/65JI 


5/22% 


3/13? 


23/100? 



Key: High = Top Third 

Middle = Middle Third 
Low = Bottom Third 



Two conclusions can be drawn from the data. In the first place, it is 
apparent that the intensive training and the interviewing practice of at 
least 24 interviews are minimal requirements for tester certification* In 
fact, most experienced testers agree that many more interviews are necessary 
before one becomes proficient and comfortable in the role of interviewer and 
rater. In the second place, it is obvious that rating is more problematic 
than interviewing. The distinctions within and across levels require a 
thorough understanding of the scale that only comes with experience in using 
it. The discussion of Question 8 below addresses several aspects of this 
point. 

Since a tester certification procedure now exists and is being 
implemented by ACTFL/ETS, a more definitive answer to this question will be 
possible in future workshops. Participants who complete the initial training 
and practicum now submit tapes and ratings to ACTFL for immediate validation. 
As a result of this process, data can be gathered to determine immediately the 
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rate of success as well as the kind of supplementary training needed. 
•Question 2. What demands^ ia terms ^ time needed for preparation. 
administration, and scoring ^ does use the oral proficiency interview make 
WPn professors ? 

The amount of time required to conduct an oral proficiency interview, 
which is of necessity an individual process, is a matter of legitimate 
concern. Table 2 (see next page) summarizes the reported time spent by 
participants in preparation for, administration, of and rating of the 
interview. 

The data reveal that beginning interviewers spent an average of 56 
minutes; (1) arranging for (13 minutes), (2) conducting (22 minutes), and (3) 
reviewing (21 minutes) the interview. Almost a quarter of this time was 
devoted to obtaining the specific type of interviewees needed for the project, 
with the remaining three quarters was spent in interviewing and rating the 
taped interview. 

As testers gain experience in choosing appropriate questioning 
techniques, the amount of time needed to obtain a ratable sample is reduced 
considerably. While there are no set time limits, a Novice-level interview 
generally lasts from 5-7 minutes, an Intermediate-level about 10-12 minutes 
and an Advanced-level or Superior-level from 15-20 minutes at the most. 
Taping the interview provides a record that can be used for delayed rating or 
to check a rating determined immediately upon completion of the interview. 
Participants in the study noted a reduction in their own time spent 
interviewing and rating as they became more practiced over the course of the 
16 project interviews. 

Participants in the project overwhelmingly (90%) considered the time 
spent on preparing, interviewing, and rating worthwhile with respect to their 
own professional development. A large percentage (845f) also believed that 
the experience was a worthwhile one for students as well. Among the 
suggestions for making more efficient use of time were: distributing an 
explanation of the process to students before the interview; rating either 




Table 2 

Participant Time Report for Preparation, Administration, and 
Rating of Oral Interviews 



Participant Mean PREPARATION Mean INTERVIEW Mean RATING 

ID Number time per interview time per interview time per interview 

ns23 



1 


3 minutes 


23 minutes 


23 minutes 


2 


little or none 


20 


20 


i| 


4 


24 


23 


5 


3 


24 


8 


6 


10 


21 


26 


7 


no data 


no data 


no data 


9 


3 


18 


8 


10 


20 


30 


30 


11 


25 


16 


40 


12 


32 


30 


30 


13 


6 


20 


15 


U 


28 


30 


34 


15 


.13 


21 


19 


16 


7 


25 


3 


18 


no data 


26 


28 


19 


no data 


no data 


no date: 


21 


10 


17 


18 


22 


no data 


no data 


no data 


23 


11 


18 


11 


24 


no data 


18 


15 


25 


22 


24 


i2 


26 


15 


22 


22 


27 


8 


21 


19 


Mean times 
for group 


13 minutes 


22 minutes 


21 minutes 



Total mean time for preparation, interview, and rating: 56 minutes per 
interview 
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during or right after Interviewing; using a check list for each level; 
formulating possible questions, probes and situations before the Interview; 
and shortening Interviews, especially at the novice level. 

Half of the respondents Indicated that time Is an Inhibiting factor when 
considering use of the oral proficiency Interview In classes of more than ten 
students. They believed that colleagues would not accept the burden of 
additional time required for the interview, especially if it were not directly 
related to the work of the course. 

•Question 3. What special conditionSf facilities and equipment do professors 
need jisfi the oral proflolenQV Interview? 

Participeuits did not Indicate a need for special conditions, facilities 
and equipment, i.e., those not normally available in the department, for 
conducting the interviews. 

TABLE 3 

Conditions, Facilities, and Equipment Needed for Oral Interviews 

I. Site used (n=22) 

Professor •s office 16/73J 
Language Laboratory 6/27% 

II. Equipment (n=l3) 

Portable tape recorder 15/83? 
Professional-type equipment 3/17? 

III. Problems in arranging/ acquiring site/ equipment (n = 22) 
None 22/IOOj 

IV. Problems in obtaining clear recordings (n = 16) 

Background noise 6 

Poor positioning of csuididate/ equipment 4 

Inexpensive equipment/ tapes 4 

Timidity of students 3 

V. Suggestions for obtaining clearer recordings 

Better equipment ("get the best") 

Clean recording heads before taping 

Use high quality new tapes 

Use soundproof room 

Place microphones carefully 

Use lavaller microphones 

Record on "neutral" territory 

Have clock visible for interviewer 
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A3 noted in Table 3, three quarters of the participants used their own 
offices to conduct the interviews while the rest used the language laboratory* 
Portable tape recorders proved adequate, though some professors reported 
problems with backgrcond noise and low-quality tapes. Students did not appear 
to be negatively affected by the presence of the tape recorder* It was noted 
that it is important to make sure that there are no telephone or similar 
interruptions during the course of the interview. 

•Question 4* What Imasit does the oral proficiency interview hkm AlCfia the 
Gurrioulum. the inatruQtional materials, and the procedures used achieve 

iM £fiala sL course? 

Tables 4 and 5 (see next pages) respectively summarize the 
data with regard to the impact of oral proficiency testing on (1) the specific 
course from which participants selected students for the project interviews 
and (2) courses in general and the overall language program. 

The primary impact was perceived to be more emphasis on developing 
communicative skills by using (l) texts that emphasize oral proficiency goals, 
(2) task-oriented instructional activities (especially "situations" and small 
group conversations), and (3) authentic recorded materials in class. It was 
noted by a third of the participants (the l£u?gest percentage of consensus) 
that the interview has more effect on materials and techniques than on goals 
and curriculum development. A logical inference from this observation is that 
the principles and procedures of the oral proficiency interview provide a 
means of realizing already existing but previously unfulfilled oral goals in 
the curriculum. 

Other areas of impact, as seen in Table 5, included using the oral 
proficiency interview and rating techniques (1) for placement and/or waiver 
purposes or as part of the student»s permanent record, (2) to establish 
expected performance levels in courses, (3) to reinforce the importance of 
oral production, thereby increasing students motivation, and (4) for 
diagnostic rather than grading purposes. 
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Table 4 

Impact of Oral Proficiency. Testing on Curriculum, Instructional Materials, 
and Procedures Used to Achieve Course Goals 

n = 23. All participamts did not answer all questions 

I. Goals and curriculum development 
A, Positive 



1 • More emphasis on communicating, especially speaking, 

in real situations 7 

2. Relate course objectives to proficiency definitions 

and grammar grids 2 

3« Already emphasizing oral skills 7 

B. Negative 

1 • Conflict with implications of Krashen*s "input 

hypothesis" (anphasis on listening comprehension) 2 

2, No intention of changing course to fit test 2 

3. Classroom language teaching has little to do with 
language proficiency 1 

II. Instructional materials 

A. Positive 

1. Develop situations and conversation cards 3 

2. Look for texts that relate to oral proficiency goals 

and techniques 2 

3. Use more authentic recorded material in class 4 

B, No opinion 10 

III. Instructional techniques 

A, Positive 

1. Students ask more questions 2 

2. Use task-oriented instructional techniques to 

simulate real situations, encourage creativity 3 

3. More group work, less drill 4 

B. No opinion 10 

IV. Testing 

A. Plan to incorporate in some way 10 

B. Other 8 

C. No opinion 4 
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Table 5 



Function of Oral Proficiency Testing in Courses and Programs 
I. Function in specific courses 



A» Measure student progress (for/ not for grade) 5 

B. Motivate students to develop oral skills 5 

C. Diagnose strengths and weaknesses 2 

D. Provide a standard for proficiency and for course design 2 
Provide opportunity to use target language in a one-on-one 
situation for twenty minutes 3 

II. Function in total language program 

A« Placement purposes 11 

B. Proficiency requir«nent for (1) language requirement, 

(2) graduating majors and minors 6 

C. Motivation factor to make students aware of importance of 

oral proficiency 4 

D. Proficiency ratings for transcript, CV 4 

E. Measure student progress as a result of a JYA experience 2 

III. Colleague Reaction 

A. Interested ("definitely" to "mildly") 14/61^ 

B. Varied ("some yes, others no") 3/^3% 

C. Not interested ("no" to ^threatened") 6/26$ 

D. Blank (all colleagues involved in PROJECT OPT) 



•Question 5. i^at oreaage ^ context variables j^e asaoQiated with 
gUQQeggfuJr SL l&e oral profioiency Interview ? 

This question was designed to determine what relationships, if any, exist 
between (1) teacher characteristics, professional training, and experience and 
(2) success in interviewing and rating. Because of the small sample size in 
each of the four languages represented in the study (French, 7; German, 4; 
Russian, 4; and Spanish, 9) the analysis was not carried out. Experience has 
shown, however, that interviewers should themselves be at Level 3, and 
preferably 3+, on the Interagency Language Roundtable scale in order to obtain 
a ratable sample and make accurate rating decisions along the full range of 
the ACTFL/ETS rating scale in acad«nic situations. 

•Question 6. MML problems iJo professors encounter when using the oral 
proficiencv interview ia ^ foreign language course ? 

Participants were asked to describe the problems they experienced in 
conducting and rating the interview. A summary of the data is found in Table 
6. 



ERLC 



24 

30 



Table 6 

Problems Encountered in Interviewing and Rating 

Percentages indicate proportion of participants who identified the item as a 
problem. 

!• Interviewing 

a. Relative difficulty of the phases of the interview. 



Warm-up i\2% 

Level Check ^ 56% 

Probes 65% 

Wind-down 32?; 

b. Acting as interviewer, not teacher 67% 

c. Conducting consecutive interviews 7651 

d. Interviewer »s own proficiency 30JI 

II. Rating 

a. Clarity of ACTFL/ETS proficiency descriptions 57% 

b. Concern for ratability of sample 57% 



III. Other problems reported 

a. Concern for time spent 

b. Role of dialect in assigning rating 

c. Use of situations (timing and suitability) 

The three most frequently mentioned problems concerned (1) assuming the 
role of tester in the interview; (2) using probes appropriately in the 
interview; and (3) assigning an appropriate rating, especially in the 
Intermediate range. 

(1) There are significant differences between the role of teacher and the 
role of tester. In the classroom, langusige teachers use several typical 
behaviors to encourage and reinforce learning. These include cueing 
responses, prompting students who hesitate, repeating student responses for 
correction or reinforcement purposes, and reacting with evaluative comments 
especially when students respond correctly. In the interview, language 
testers must avoid these kinds of behaviors for a very specific reason: 
obtaining a ratable sample of the student^s language proficiency depends upon 
an objective approach to interviewing that requires that the student alone 
show what he/she can and cannot do. Prompting, repeating, and encouraging by 
means of judgmental feedback on the part of the interviewer do not allow the 
student to do this. Thus, teacher-testers must learn to wait for answers 
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without promptlzig, accept what students say without correcting or evaluating 
what Is saldi and permit students to reach the point where they can no longer 
express themselves comfortably* They must do these things, however, In a non- 
threatening manner that does not Intimidate. Teacher-testers report that It 
Is most difficult to accomplish this when Interviewing one's own students 
(See Question 8 below). 

(2) In the Interview, probes are used to find out whether or not the 
speaker can go beyond the estimated level of proficiency. They serve to 
establish the "celling" or highest level at which the speaker can function 
consistently by taking the speaker to "linguistic breakdown," a point at which 
the speaker can no longer convey meaning effectively. An Interview should 
Include at least three probes In order to show conclusively that the speaker 
cannot sustain speech at a higher level. Beginning testers often feel awkward 
putting speakers In such an uncomfortable position once, let alone three times 
during an Interview. They also are not sure of the appropriateness of their 
probes and may skip a level with the result that the speaker Is not tested at 
a level which he/she might have been able to manage. 

(3) Rating the Interview requires considerable familiarity with the 
ACTFL/ETS rating scale and the distinctions between and within levels. In 
the first place, It Is extremely Important that raters recognize the 
characteristics of each of the four base levels (Novice, Intermediate, 
Advanced, and Superior) in the system. Errors In rating decisions across 
these major borders Indicate significant problems with either the sample or 
the matching of the sample to the definitions. In the second place, it is 
important that raters understand the different degrees of proficiency (low, 
mid, high) within the levels. While errors in assigning these ratings within 
levels are not as critical as those across levels, they are a matter of 
concern and should be corrected with additional practice. It should be noted, 
however, that the ACTFL/ETS definitions were, at the time of the study, still 
being revised and some were acknowledged to need revision to eliminate 
possible ambiguities such as those reported by participants. 



In addition to the three principal problems, some participants noted that 
their own proficiency was a matter of concern and that^ after conducting more 
than three Interviews In one session, they experienced considerable fatigue 
and began to repeat questions. 

^Question 7. 1^ further ^r^jLning la. necessary help professors test oral 
proficiency In their courses? 

Participants In the study agreed that the training sessions and the 24 
critiqued interviews provided them with a basic understanding of the 
principles and procedures of the evaluat.-^^on process. They also acknowledged 
that increased confidence £Uid accuracy in interviewing and rating are a 
function of experience. They recommended more observation of live or 
videotaped interviews and additional training, through shorter workshops, 
especially to determine plus points in level. The need for effective 
interpersonal skills was also identified. (See Table 7.) 



1. Practice in: 

A. Interviewing 

1. observe live/video taped Interviews (4) 

2. conduct interviews with partner (2) 

3. be interviewed (1) 

4. practice in varying questions (1) 

5. interpersonal skills (1) 

B. Hating 

1. to determine plus points in level (2) 

2. need critique (1) 

3. more controlled evaluation of results (1) 

2. Other: 



The evaluators who critiqued the participant tapes Indicated a need to 
provide more training in estimating proficiency during the Interview to make 
appropriate choices of interviewing techniques such as probes and situations. 
(See Table 8.) 



Table 7 



Perceived Further Training Needs (Self-evaluation) 



1. 
2. 

3. 
4. 



more time to prepare and do interviews (5) 
a course or additional three-day workshop (2) 
clear descriptions of novice-high to advanced (1) 
become a certified trainer of testers (1) 
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Table 8 

Evaluator's Analysis of Participant Interviewing Performance and 

Need for Further Training 

!• AnsuLysis of Participant Interviewing Performance 

Strengths: 

High occurrence of behaviors: 

Treated candidates in friendly manner and spoke at normal speed 
Gave candidates feeling of accomplishment 
Allowed candidates to talk without interruption 

Average occurrence of behaviors: 

Made smooth transitions from one phase to another 

Stayed with topics long enough 

Took candidate to highest level of proficiency 

Weaknesses: 

Inaccurate estimate of level during interview 

Insufficient use of probes and situations at appropriate levels 

Excessive use of yes/no questions 

Lack of wind«-down 

II. Areas for further training 

Estimating proficiency during interview to make appropriate choices of 

question type and content 
Practice in using probes and situations 

Development of thoroughness in interviewing to assure a ratable sample 
that clearly demonstrates what a candidate can and cannot do 



They also identified the excessive use of yes/no questions as an area of 
concern and noted that participants need to sample several content areaSf 
especially as the interviewee»s level of proficiency increases. The ultimate 
goal is a thoroughness that assures a ratable sample that clearly demonstrates 
what a candidate can and cannot do when speaking the foreign language. 
•Question 8. W^at differenQea^ 1£ anv, appear in i&g Interviewlnfl^ and rating 
teotiplques sL Prpf^ggprs whep Intervlewlni; their .sim students ^ students 
whom they jifi jifit know ? 

A critical question is whether or not professors can obtain valid and 
reliable ratings when interviewing their own students. In many foreign 
language departments, the number of faculty members teaching a particular 
language may be limited to one or two professors. Thus, if students are to be 
interviewed and rated, it must be done by someone who is already familiar with 
what those students have done in class. The study provided data regarding the 
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participants* perceptions of rating their own students and the evaluators* 
perceptions. 

While acknowledging that Interviewing one's own students Is more 
difficult than Interviewing those one does not know, over two thirds of the 
participants In the study believed that Interviews with their own students 
were as valid and reliable as those with unknown students (See Table 9). 
Twice as many (6 of 23) felt more comfortable with their own students than 
with unknown students (3); however, a similar number (7) noted that they had 
to resist slanting the Interview toward known vocabulary and grammar. A third 
of the participants were surprised at the differences between the ratings and 
class performance with the usual expectation being a higher rating. 

The evaluators who critiqued the participant project tapes completed 
Questionnaire III in which they provided data on the interviewer elicitatlon 
and rating techniques of known and unknown interviewees. Data were also 
analyzed on the degree of agreement between the participant ratings and the 
evaluator ratings of the four randomly-selected project tapes that were 
critiqued by the evaluators. A summary of the comparison of ratings is found 
in Table 10. Additional tables containing the data from which the comparisons 
were drawn are found in Appendix C. 

The evaluators who critiqued the participant tapes affirmed that, 
regardless of whether or not the professors were interviewing known or unknown 
students, 95% of the interviews constituted ratable samples. This finding 
Indicates that, in the opinion of these experienced raters, it is possible to 
Interview one's own students in a valid and a reliable manner by objectively 
following the procedures for conducting the four phases of the Interview. 

Rating the interview proved to be more problematic than conducting a 
rating interview. Based on a stratified sample of all the interviews, the 
evaluators agreed with 39? of the participants' ratings and disagreed with 
61$. When they disagreed, they raised the rating 47? of the time and lowered 
it 53? of the time. In 82? of the cases of disagreement, the magnitude was 
one step in the rating scale (from a low to mid, for example). However, in 



Table 9 

Participant Perceptions of Differences in Interviewing and Rating 

Known and Unknown Students 



1. Differences in your own behavior/ feelings while interviewing 
known/unknown candidates. 

More patient, easier with others 3 

More relaxed with own 6 

Presumed too much about own 1 
Had to reaist slanting toward 

known vocabulary and grammar 4 

Other 5 

None/Blank 2 

2. Differences in student behavior/ feelings. 

Own students more nervous 5 

Other students more nervous 4 

Other 2 

None/Blank ^ 8 

3. Differences in own behavior/ feelings while rating known/ unknown students. 

Surprised at difference between 

class and interview (usually 

expected higher rating) 7 

Other 5 

None/Blank 9 

4. Are your interviews with your own students as valid and reliable as those 
with unknown students? 

Yes 15 

No 2 

Not sure 4 

Blank 1 

Difficult to interview own 4 

Difficult to rate own 2 

5. Is it possible for interviewer to rate and conduct oral proficiency of 
own students in valid and reliable manner? 

Yes 15 

No 0 

Blank 8 

6. What can evaluator do to get valid and reliable interviews? 

Be objective and follow guide- 
lines 13 
Have colleague sit in 4 
Blank 3 
Other 3 
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Table 10 

Comparison of Interviewer and Evaluator Ratings 
5 interviews in 92 (5%) judged to be unratable . 

1. Index of agreement/disagreement: 39/6 IJt 

2. When evaluator s disagreed with interviewer rating, evaluator: 

Raised rating: U7% 
Lowered rating: 53% 

3. Magnitude of disagreement: 

1 rating: Q2% 

2 ratings: ^Q% 

4. Rating own students/other students: 

Own students: 

Evaluator agreed: 26% 
disagreed: 74^ 

When disagreed, evaluator: 

raised rating: i\i\% 

lowered rating: 4751 

non-ratable: S% 

Other students: 

Evaluator agreed: 5k% 
disagreed: i\6% 

When disagreed, evaluator: 

raised rating: 3 851 

lowered rating: 52% 
non-ratable: ^0% 

the 61^ of the cases in which there was disagreement, kS% of the discrepancies 
were across major boundaries (from Intermediate to Advanced, for example) and 
5151 were within the same level (from Intermediate-mid to Intermediate-high, 
for example). In the kS% ot cases of discrepancies across major boundaries, 
2k% occurred from Novice to Intermediate, 52% from Intermediate to Advanced, 
and 2451 from Advanced to Superior. 

These findings indicate that beginning raters are likely to agree with 
experienced evaluators on two out of five interviews. It is interesting to 
note that the recently implemented ACTFL/ETS criteria for certification as an 
oral proficiency rater state that "the ratings of the applicant and trainer 
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must correspond precisely on a minimum of two of the five interviews." 

The findings also indicate that when beginning and experienced raters 
disagree, the magnitude of disagreement is likely to be one step in Tour out 
of five cases and two or more in the remaining case* Again comparing this 
finding with the ACTFL/ETS criteria, a range of one step is permissible, for 
certification purposes, in three of the five interviews submitted for 
evaluation. However, the one step discrepancy must be within the same base 
level and not across a major boundary. The data for study indicated that when 
beginning and experienced raters disagree, the disagreement is likely to cross 
a major boundary in one out of two interviews. Furthermore, at least half of 
the time, the disiigreement is found in the Intermediate to Advanced range of 
the scale. These findings have two important implications for the training 
of oral proficiency testers: (1) beginning testers need more practice in 
recognizing the differences between the Intermeillate and Advanced levels of 
the scale, and (2) additional definition is needed in the scale itself to make 
more apparent the differences between the two levels. 

When the data were analyzed to determine differences in rating known and 
unknown students, the evaluators agreed with the rating of known students 26% 
of the time and with unknown students 5^% of the time. With more than twice 
the probability of agreement between experienced and inexperienced raters on 
rating unknown students, it appears that factors associated with previous 
classroom contact with students do affect a rating given by beginning testers. 
It is interesting to note, however, that when evaluators disagreed with 
ratings, they raised them of the time for known students but only 38% of 
the time for unknown students. It would seem that professors might be 
compensating for their experience with known students by giving lower ratings. 

The statistics support the participant consensus that it is more 
difficult for beginning testers to rate their own students than to rate 
students who have not studied with them. A reasonable solution to the dilemma 
is to have students interviewed by qualified testers, even if the student has 
studied with the interviewer, and then have the tapes rated by independent 



evaluators. When a larger pool of certified testers is available, it will be 
interesting to continue the study of the validity and reliability of rating 
one's own students. 

•Question 9- Whftfr ProflOlgPOY l?Y9lg a8990l»t9d wAth tvploal levels 
iPStruOtloP (h^ffinnlnKt intermediate, advanced) ^ ^ college level ? 

Participants in the study interviewed students enrolled in beginning, 
intermediate, and advanced language classes and in literature and culture 
classes taught in the foreign language. The resultant data (See Table 11) 
provides some preliminary information on expected levels of proficiency in 
each of the typical levels of instruction. It should be noted that the sample 
(beginning students = 35, intermediate students = 71, advanced students = 28, 
literature and culture students = 20) is small and includes all four languages 
(French, German, Russian, and Spanish) of the project. In addition, the 
ratings were determined by beginning testers involved in a training project. 
As discussed above, experienced evaluators disagreed with these ratings 6M of 
the time and lowered the rating 53% of the time. Half of these disagreements 
involved major boundary decisions (from Intermediate to Advanced, for example) 
and half involved decisions within levels (from mid to high, for example). 

Based on the data reported in Table 11 (see next page), the mean 
proficiency levels in each level of instruction were: 

1. Beginning language courses (1st and 2nd semester) 



Spanish (n = 20) : 



Novice High 



Intensive Russian (n = 15): 



Intermediate Mid 



2. 



Intermediate language courses (3rd and 4th semester) 



French (n = 32) : 



Intermediate Mid 



German (n = 24) : 



Advanced 



Russian (n = 15) : 



Intermediate Mid 



3. 



Advanced language courses (5th and 6th semester) 



French (n = 8): 



Intermediate High 



Spanish (n = 20) : 



Intermediate Mid 
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Table 11 

Relationship of Proficiency Level to Course Level 



Course/Language 


n 


range 


X 


mode/n 


Beg. French 










Beg. German 










Beg. Russian (Intensive) 


15 


NH-IH 


IM 


IM/6 


Beg. Spanish 


20 


NL-IH 


NH 


NM/8 


xnu • r rencn 


32 




in 


IM/12 


Int. German 


24 


"FT CT 

IL-S 


A 


IH/o 


inu. nussian 


15 


NH-IH 


In 


IM/o 


Int. Spanish 










Adv. French 


o 
0 


IL-S 


IH 


IH/3 


Adv • German 










Adv. Russian 










Adv« Spanish 


20 


IL-A+ 


IM 


IH/7 


French Lit. /Cult. 


8 


IH-A+ 


IH 


A/ 5 


German Lit ./Cult. 










Russian Lit. /Cult. 










Spanish Lit. /Cult. 


12 


IM-A+ 


IH 


IH/6 



Key: 1. Beginning = enrolled In first/ second s^ester language course 

2. Intermediate = enrolled In third/ fourth semester lauiguage course 

3. Advanced = enrolled In fifth semester language course 

4. Llt./Cult. = enrollad In literature or culture course taught In a 

foreign language 



4. Literature and Culture courses 

French (n = 8): Intermediate High 

Spanish (n = 12): Intermediate High 

While It may seem natural to equate level of course with level of 
proficiency^ this tendency must be avoided, especially as the level of 
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instruction advances. Tentative as the data is, it does reflect an important 
characteristic of the ACTFL/ETS scale: the ranges on the scale are not fixed 
increments on an absolute scale but rather increasingly larger segments that 
require progressively greater command of function, content, and accuracy of 
language use. Thus, while the Beginning Spanish students were rated 
predictibly in the Novice range, all the other students, with the exception of 
the Intermediate German students, were rated in the Intermediate range. The 
fact that 48 of these students were enrolled in what are, for instructional 
purposes, called "advanced" courses does not mean that they demonstrate 
proficiency at the Advanced levels. 

As more testers are certified and more students are evaluated using the 
oral proficiency interview, statistics related to the relationship of course 
level and proficiency level will be more reliable. Such statistics will be 
useful to departments in colleges and universities in determining how their 
students compare, in terms of proficiency, with students at similar levels of 
instruction in other institutions, 

S^mpary results. To summarize the analysis of the data generated for 
the study, the following answers can be given for the nine research questions: 

1 . Professors can learn to conduct and rate oral proficiency interviews 
effectively after completing an intensive four-day workshop and a 
supervised practicum that includes a critique of 24 interviews. It 
should be noted, however, that considerable additional practice is 
generally necessary to become a truly proficient and comfortable 
interviewer and rater. 

2. Initially, testers spend approximately 40 minutes per interview and 
rating sessions. Tliese times can be expected to decrease as the 
tester becomes more familiar with the procedures. The length of 
time spent also depends upon the level of proficiency of the person 
being interviewed since higher levels require longer interviews. 

3. Facilities such as those normally available to college professors 
are sufficient to obtain ratable interviews. 



ERIC 



35 

41 



4. The oral proficiency interview can have a significant impact on 
materials and techniques used in foreign language courses. It is 
also useful as a motivational factor to reinforce the Importance of 
oral goals and objectives. 

5. The question of what presage and context variables are associated 
with successful use of the oral proficiency interview was not 
answered in the study. Experience has shown, however, that 
interviewers should themselves be at least at level 3+ on the ILR 
scale. 

6. Problems most frequently encountered by novice testers include; (1) 
assuming the role of tester, not teacher, in the interview; (2) 
using probes appropriately in the interview; and (3) assigning an 
appropriate rating, especially in the intermediate range. 

7. Further training needs identified by novice testers and the 
evaluators who critiqued their interviews include more observation 
of live and taped interviews, short workshops to practice rating, 
and practice in the incorporation of a range of techniques and 
topics in the interview to assure a ratable sample. 

8. Both the participants in the study and the evaluators agreed that it 
is possible to interview one's own students in a valid and reliable 
manner. With respect to rating the interviews, it was found that 
the ratings of beginning testers were identical to those of 
experienced evaluators 40> of the time. This percentage meets the 
recently implemented ACTFL/ETS criteria for certification as an oral 
proficiency rater. It should be noted that differences should not 
cross major boundaries from one level to another as was the case 50% 
of the time in this study. 

9. Mean proficiency ratings were in the Novice range for students 
enrolled in "Beginning" language courses and Intermediate for 
students enrolled in "Intermediate" and "Advanced" language courses 
as well as in literature and culture courses. The data support the 
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contention that the tendency to equate proficiency level with course 
level is not a valid one. 



Chapter Four 

THE MATERIALS DEVELOPMENT AND DISSEMINATION COMPONENT 



The third component of PROJECT OPT consisted of the development and 
dissemination of materials pertaining to oral proficiency testing. In 
addition to publications, this component included workshops and papers given 
as a result of the project. 
POBLICATIQNS 

To date, the project has produced three publications on the topic of oral 
proficiency testing. Copies of the publications are included with this 
report. 

1. Qr^ Proficiency Testing In College^Level Forelffn l^^nffy^^g^ Programs, 
A Han<ibQ<?k IsL Foreign Language Departments . 

This handbook is designed to acquaint foreign language departments in 
colleges and universities in the United States with recent developments in 
oral proficiency testing in foreign language programs. It provides an 
overview of the rationale, development, procedures, and implications of oral 
proficiency evaluation in the context of the teaching mission of college and 
university foreign language departments. In addition, it reports the findings 
of PROJECT OPT. The handbook is not meant to be a training manual but rather 
an introduction to the principles and techniques of oral proficiency testing. 

The handbook was published by the American Council on the Teaching of 
Foreign Languages with funds from the project. Through an agreement with 
C. Edward Scebold, Executive Director of ACTFL, chairpersons of foreign 
language departments that are members of the Association of Departments of 
Foreign Languages in four-year colleges and universities of the United States 
were invited to reserve a free complementary copy. A total of approximately 
700 copies were requested and distributed by the ACTFL Materials Center in 
May, 1985. 

2. Qr^l Proficlencv Testing In Russian, A Handbook. 

This 35-page handbook was compiled by the directors of PROJECT OPT for 
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use in a one-day familiarization workshop to acquaint Russian professors with 
the principles and procedures of oral proficiency assessment. It was used for 
the first time at a meeting of the Northern New England Chapter of the 
American Association of Teachers of Slavic and East European Languages 
(AATSEEL) on October 15, 1983. Funds for the handbook and the workshop were 
provided by the Central University Research Fund of the University of New 
Hampshire. 

The handbook, which is the first publication of its kind for teachers of 
Russian in academic settings, provides an overview of recent developments in 
oral proficiency testing in general and treats specific aspects of oral 
proficiency testing in Russian. Like the handbook published by ACTFL, this 
volume is intended to be an introduction to principles and techniques and is 
not a training manual. A limited number of the original edition of 200 copies 
is available from Dr. Aleksandra Fleszar (Mayewski), Associate Director of 
PROJECT OPT, Department of German and Russian, Murkland Hall, University of 
New Hampshire, Durham, NH 03824. The handbook is also available on microfiche 
from the Educational Resources Information Center (ERIC ED 240 869). 

3. Publication of PmiEgT 

During the second year of the project (1983-4), two issues of POPT y a 
newsletter for participants and friends of PROJECT OPT, were published and 
disseminated (See Appendix D). This newsletter served as the primary means of 
communication among the members of the Northern New England network of college 
and university faculty who are currently working with oral proficiency 
testing. 

Although funding for PROJECT OPT has ended, the Directors will continue 
to serve as facilitators for the network and will publish, when possible, 
additional issues of POPT to enable network members to maintain contact and to 
encourage further use and development of oral proficiency evaluation 
procedures in the region. 

4. Further publications. In addition to the three publications already 
disseminated, the directors are preparing an article on the research component 



of PROJECT OPT for submission to a national journal on foreign language 
pedagogy • 

WORKSHOPS m num. 

PROJECT OPT also resulted in the presentation of workshops and papers on 
the topic of oral proficiency testing. A summ€U?y of these activities follows: 

1. Workshops and papers presented by Dr. Barbara H. Wing, Director of 
PROJECT OPT. 

In response to requests from secondary and post-secondary level foreign 
language educators and supervisors, Dr. Wing presented the following 
workshops during the grant periods 

a. A Workshop for Foreign Language Teachers in the Nashua, NH, School 
System. 25 March 1983. 

b. A Workshop for Foreign Language Teachers in the New Hampshire 
Seacoast Region. 11 May 1983. 

c. A Workshop for French Teachers of Immersion Classes at Nippissing 
University College in North Bay, Ontario, Canada. 12 July 1983. 

d. A Workshop for Foreign Language Teachers at Rivier College, Nashua 
NH. 11 July 1984. 

e. Consultant in Spanish for A Workshop on Elicitation Techniques, a 
pre-conference workshop at the 1984 Northeast Conference on the 
Teaching of Foreign Languages on April 14 in New York City. 

Dr. Wing also presented a paper on PROJECT OPT on November 23, 1983, at 
the 1983 Annual Meeting of the American Council on the Teaching of Foreign 
Languages in San Francisco. 

2. Workshops and papers presented by Dr. Aleksandra Fleszar (Mayewski), 
Associate Director of PROJECT OPT. 

1 . A Familiarization Workshop in Russian for the Northern New England 
Chapter of the Association of Teachers of Slavic and East European 
Languages. University of New Hampshire, 15 October 1983. 

2. "Oral Proficiency Testing: Results and Applications to Overseas 
Programs." Paper presented at the Annual Meeting of the American. 



Association of Teachers of Slavic and East European Languages. New 
York, December 1983. 

"Assessing Language Proficiency Levels." Chairperson of panel 
discussion at the Annual Meeting of the American Association of 
Teachers of Slavic and East European Languages. Washington, 
December, 1984. 

Member of the AATSEEL Committee on Testing and Professional 
Development. 1984-85. 
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Chapter Five 
CONaUSIONS 

PROJECT OPT, a research and training project, was designed to achieve, 
three objectives: 

1 . to determine the feasibility of incorporating specific techniques of 
oral proficiency evaluation in foreign language courses at the 
college level; 

2. to create a network of foreign lemguage professors in northern New 
England colleges and universities trained in the use of oral 
proficiency evaluation procedures; 

3. to develop a handbook of oral proficiency evaluation techniques for 
college-level foreign language faculty. 

Answers to the nine research questions investigated in the project 
indicate that it is feasible to use the ACTFL/ETS Oral Proficiency Interview 
in foreign language courses at the college and university levels. 
Participants reported benefits that included a more objective evaluation of 
oral skills, the potential for increased student motivation, and a positive 
impact on the development of curriculum, materials, and instructional 
techniques in courses. They also pointed out that the process may be even 
more useful at the departmental program level where important decisions, such 
as certifying for transcript or professional reasons, are involved. Problems 
encountered in implementing oral proficiency evaluation techniques were deemed 
to be resolvable. The most pressing question, that of ensuring sufficient 
numbers of certified testers with adequate time for conducting and rating 
interviews, will require the development of creative solutions, such as use of 
networks to provide testers for the institutions within the region. 

There is a network in place in northern New England that will continue to 
function although funding for the project has ceased. At least one proposal 
has been submitted to the Fund for the Improvement of Post Secondary Education 
(FIPSE) that will expand the network through training workshops at the 
University of New Hampshire during the summers of 1986 and 1987. These 
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workshops are part of a larger effort to incorporate oral proficiency testing 
into the Program in International Perspectives, an interdepartmental effort, 
at the university. The proposal also includes funding for foreign language 
instruction for professors to improve their proficiency in languages that are 
related to their special areas of expertise. 

In addition to the handbook. Oral Profioiencv Testing in College"Level 
Foreign Languai^e Programsy the project produced the handbook for Russian 
professors and was the subject of papers and workshops at national and 
regional meetings. It is likely that over 1000 people were reached, in one way 
or another, by the various activities and products of the project. 

PROJECT OPT provided answers to questions that are representative of what 
faculty members, in particular, often ask about the procedures and results of 
oral proficiency evaluation. During the project, additional questions were 
raised that should be the object of future investigations. Some of these 
questions are: 

!• How does oral profloienGV testing, which represents a global 

evaluation of functional language use, relate to achievement testing 
of the oral skills, which relates to how effectively students have 
learned a particular corpus of material within a specified period of 
time? 

2. What revisions are needed in the ACTFL/ETS descriptions of levels to 
provide for higher levels of interrater reliability upon completion 
of the trairing period? 

3. How has incorporation of oral proficiency testing actually affected 
courses and programs now that the procedures have been used in 
several colleges and universities around the country for three 
years? 

4 . How does the concept of proflQienov. which has been suggested as an 
organizing principle for the teaching of foreign languages, affect 
the development of curriculum, materials, and instructional 
techniques for the other skills of listening, reading, and writing. 
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and the development of cultural sensitivity and understanding? 
5. How can training, interviewing, and rating services be provided most 
efficiently and economically to faculties and students in post- 
secondary and secondary institutions in the United States in order 
to link together the several networks that already exist? 
Answers to these questions are being formulated on campuses around the 
country and in agencies like the American Council an the Teaching of Foreign 
Languages and the Educational Testing Service, The growing number of informed 
and trained testers is an indication of the responsibility that foreign 
language teachers assume for their own and their students' accountability. It 
represents what has been a continuing commitment of the foreign language 
teaching profession: the improvement of the quality of teaching and learning 
in their courses and their programs. 
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Appendix A: 

Directories of Workshop Personnel and Participants 

1. Staff Directory 

2. Project Tape Evaluators 
August Workshop Consultants 

3. Participant Directory 
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PROJECT OPT 



UNIVERSITY OF NEW HAMPSHIRE D.P«,«« Anci.nc.nd Mod.n Un,u^.„™, ut.r.tur« 

DURHAM. NEW HAMPSHIRE 03824 Coii.wi of ub^i Art. nout.r.turt. 

Murkland Halt 



Staff Directory 



Dlrectop 

Barbara H. Wing 
AMLL/Murkland 209C 
University of New Hampshire 
Durham, NH 03824 
Spanish (603) 862-1218 



Asaoolata Director 

Sandi Mayewaki 
AMLL/Murkland 16A 
Univ. of New Hampshire 
Durham, NH 03 824 
Russian (603) 862-1218 



WorkghCP Coodinator 

Judith Liskin* Gasparro 

College Board Program 

Educational Testing Service 

Princeton, NJ 08540 

Spanish (609) 921-9000 ext. 1487 



Pardee Lowe, Jr. 
CIA Language School 
Washington, D.C. 20505 
German (703) 351-3065 



WgrkahoP Conaultanta 



Alice Omaggio 
Department of French 
707 South Mathews Ave. 
Univ. of Illinois / 

Urbana-Cham paign 
Urbana, IL 61801 
French (217) 333-2020 



Reaeareh Assistant 

Betty Le Compagnon 
291 Forest Park • 
Durham, NH 03824 
(603) 868-1162 
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. UNIVERSiTY OF NEW HAMPSHIRE D.p.rtm.nt of Ancient and M.a.rn Un.uac,-and Ufratur- 

DURHAM, NEW HAMPSHIRE 03824 Coiitg* of ub^rii Am 

Murkltnd Htii 



Project Tape Eraluators 



Dr. Heidi Byrnes 
914 Whann Avenue 
McLean, VA 22101 

Mrs. Maria Casey 

7861 Newport Glen Pass 

Annandale, VA 22003 

Dr. Ellen Mitchell 

201 Glenwood Circle, Apt. 213 

Monterey, CA 93940 



August Workshop Consultants 



Use Christoph 

CIA Language School 

Washington, D.C. 20505 

German 

Frank W. Medley, Jr. 
Department of Foreign Languages 
University of South Carolina 
Columbia, SC 29208 
Spanish (803 777-2791) 

Carol Murphy 

Department of Romance Languages and Literatures 
University of Florida, 170 ASB 
Gainesville, FL 32611 
French (904 392-5678) 
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UNIVERSITY OF NEW HAMPSHIRE 



DURHAM, NEW HAMPSHIRE 03824 coiitgt of ub«r«i Am 

Murktand HatI 

Participant DlreGhnrv 



0«partm«nt of Ancient and ^fod•rn Languages and Littraturtt 



Gilbert J. Albert 

University of Maine at Fort Kent 

Pleasant Street 

Fort Kent, ME 04743 

French (207) 834-3162 

Gregory W. Allar 
AMLL 

University of New Hampshire 
Durham, NH 03824 

Russian (603) 862-1218 ext. 33 

Judith Anderson Allbee 

Department of Modern Languages 

Norwich Universi ty 

Northfield, VT 05663 

Spanish (802) 4H5*5011 ext. 230 

Robert Baker 

Department of Russian 

Middle bury College 

Middlebury, VT 05753 

Russian (802) 388-3711 ext. 2535 

Jean- Pierre Berwal d 

Dept. of French and Italian 

University of Massachusetts 

Amherst, MA 01003 

French (413) 549-5029 

Thomas Beyer 

Department of Russian 

Middl ebury College 

Middlebury, VT 05753 

Russian (802) 388-3711 ext. 2532 

Ute Brandes 
Department of German 
Amherst College 
Amherst MA 01003 
German (413) 542-2481 

Arna Bronstein 
AMLL 

University of New Hampshire 
Durham, NH 03824 
Russian (603) 862-1218 



Eric A. Bye 

The Experiment in Inter national . Livi ng 
Kipling Road 
Brattleboro, VT 05301 
French (802) 257-77 51 

Beatriz C. de Fantini 

The Experiment in International Living 
Kipling Road 
Brattleboro, VT 05301 
Spanish (802) 254-8782 

Cecilie L. French 
Modern Langi' age Section 
Keene State College 
Keene, NH 03431 
German (802) 254-8566 

Virginia Garlitz 

Foreign Language Department 

Plymouth State College 

Plymouth, NH 03264 

Spanish (603) 536-1550 ext. 466 

Joel D. Goldfield 

Department of Foreign La'^guages 

Plymouth state College 

Plymouth, NH 0326 4 

French (603) 536-1550 ext. 277 

Margaret S. Langford 
Modern Language Sec t ion 
Keene State College 
Keene, NH 03431 
French (603) 357-3204 

Sister Pauline LeBel, CSG 
Department of Foreign Languages 
Notre Dame College 
Manchester, NH 03104 
Spanish (603) 622-2757 

John R. Maier 

Foreign Languages and Literatures 
Bates College 
Lewiston, ME 04240 
Spanish (207) 784-9104 



ERLC 



Grover Marshall 
AMLL 

University of New Hampshire 
Durham, NH 03824 
French (603) 862-3856 

Mark McCaffrey 

Department of Spanish and Italian 
Middle bury College 
Middlebury, VT 05753 
Spanish (802) 388-3711 

Bettye Myer 
Department of French 
Fredonia College 
Fredonia, NY 14063 
French (716) 436-5021 

Helmut Pfanner 
AMLL 

University of New Hampshire 

Durham, NH 03824 

German (603) 862-1218 ext. 42 



Marsha G. Robinson 
AMLL 

University of New Hampshire 
Durham, NH 03824 
Spanish (603) 862-1218 

Marion Sonnenf eld 

Foreign Languages and Literature 

Fredonia College 

Fredonia, NY 14063 

German (716) 673-1953 

Marina Stretch 
Modern Language Section 
Keene State College 
Keene, NH 03431 
Spanish (603) 352-1909 

John H. Turner 

Department of Romance Languages 

Bowdoin College 

Brunswick, ME 0401 1 

Spanish (207) 725-8731 ext. 363 



John D. Westlie 

Dept. of Modern Foreign Languages 

Colby College 

Waterville, ME 04901 

French (207) 873-1131 ext. 2148 
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Appendix B: 
Instruments 



1 . Questionnaire I: Participeuit Questionnaire 

2. Questionnaire II: Participant Project Report 

3. Questionnaire III: Consultant Report of 
Intarviewer Elicitation and Rating Techniques 

4. Participant Rating Sheet for Practice Interviews 

5. Participant Rating Sheet for Project Interviews 

6. Participant Project Log 

7. Evaluator Rating Sheet 
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Participant Queationnaire 

Please answer all questions by circling the letter of the one most 
appropriate answer. If you indicate o ther ^ explain briefly in the 
space provided. Read all the answers in each item before marking the 
most appropriate one. Note that the term "target language" refers to 
the foreign language you are currently teaching. 

SgQtlQQ Qng : Preparation Do not write in 

ihlA column 

1. I first learned/studied the language I am now 
teaching: 

a) informally as a child 

b) in school (elementary, junior high or high 

school ) 

c) in a college or university 

d) other i . / 



I first learned/studied the language I am now 
teaching because: 

a) I needed it to communicate with the people 
around me. 

b) I wanted to speak and understand the language. 

c) It was a requirement for my course of study. 

d) other^ 2. / 

My undergraduate major field of study was: 

a) French 

b) German 

c) Spanish 

d) Russian 

e) other ; 3. / 

As an undergraduate I: 

a) received academic credit for a study-abroad 
program in a target-language country. 

b) did not receive academic credit for a study- 
abroad program. 4. / 



As an undergraduate, I lived or travelled in a 
target language country (exclusive of study for 
credit) for a total of: 

a) more than three months. 

b) less than three months. 

c) not at all . 5 ^ 

As an undergraduate student: 

a) I had a formal course in foreign language 
teaching. 

b) I did not have a formal course in foreign 
language teaching. 5 ^ 
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7. As an undergraduate I: 

a) student-taught a foreign language. 

b) did not student « teach a foreign language. 

8. A3 a graduate student: 

a) I received academic credit for study In a 
target language country. 

b) I did not receive academic credit for study 
In a target language country. 

9. As a graduate student I lived or travelled In a 
target country (exclusive of study for credit) for 
a total of; 

a) three months or more. 

b) less than three months. 

c) not at all. 

10. As a graduate student I: 

a) had a formal course In foreign language 
teaching . 

b ) did not have a formal course In foreign 
language teaching . 

11. As a graduate student I taught a language course: 

a) regularly. 

b) occasionally. 

c) once. 

d) never. 



geql^j^Qn Two : Professional Growth 

12. In the past three years I have spent a total of 
in a target language country. 

a) no time. 

b) 1-5 weeks. 

c) 6-15 weeks. 

d) 15 weeks - 9 months. 

e) more than 9 months. 

13. I have served as a group leader for students 
on a trip to a foreign country: 

a) never. 

b) 1-3 times. 

c) 3-6 times. 

d) more than 6 times. 

14. I am a member of foreign language 

association ( s ) . 

a) state/ regional . 

b) national/ international . 

c) (a) and (b) . 

d) no. 



15. I attend state and national professional meetings: 

a) two or more times a year. 

b) once a year. 

c) every 2 or 3 years. 

d) every 5 or 6 years. 

e) almost never. 15. 



16. At professional meetings I attend a 

session on foreign language teaching methodology. 

a) always. 

b) usually. 

c) occasionally. 

d ) never . 16. 

17. I read foreign magazines or newspapers in the 
target language; 

a) weekly. 

b) monthly. 

c) quarterly. 

d) never. 

18. I read a professional journal of foreign 
language teaching: . 

a) regularly. 

b) occasionally. 

c) never. 18. 

19. Previous to this workshop I have: 

a) attended a short session on oral proficiency 
testing (1-4 hours). 

b) attended a day-long workshop. 

c) attended a workshop for more than one day. 

d) not attended any such sessions. 19. 



Section Ihr^: Curriouluig and Instruction 

Questions 20-25 are to be answered for three levels of language 
instruction (elementary, intermediate, advanced). Write the letter of 
the appropriate answer in the blank following each question number. 

20-22. In my department, there are stated oral 
objectives for language courses. 

a) yes. 

b) no. 

20. Elementary level. 20. / 

21. . Intermediate level, 21 . / 

22. Advanced level. 22. / 
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23-25. At my institution students have access to a: 

a) library-type language laboratory. 

b) a dial-access language laboratory. 

c ) no language laboratory . 

d) other laboratory facilities. 

23. Elementary. 

24. Interdemediate • 

25. Advanced. 

For the following questions, answer only for those levels that you 
have taught in the past two years. 

26-28. In the absence of departmental objectives, I 
I have developed specific oral objectives for 
my language courses. 

a) yes. 

b ) no . 

c) not applicable. 

26. Elementary. 26. / 

27. Intermediate. 27*. / 

28. Advanced. 28. / 



29-31 • In my language classes (exclusive of courses 
specifically designed for conversation), a 

student's speaking ability constitutes 

of the final course grade. 

a) 0-10 % 

b) 11-20 % 

c) 21-30 % 

d) 31-40 % 

e) more than 40 % . 

29. Elementary. 

30. Intermediate. 

31. Advanced. 

32-34. The average size of my language classes is: 

a) 1-8 students . 

b) 9-16 students . 

c) 17-24 students. 

d) 25-30 students. 

e) more than 30 students. 

32 . Elementary 

33- Intermediate. 

34. Advanced. 
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35-37. In my opinion, a reasonable class size to 
develop speaking skills is: 

a) 1-8 students. 

b) 9-16 students. 

c) 17-24 students. 

d) 25-32 students. 

e) more than 32 students. 

35. Elementary. 

36 . Intermediate . 

37. Advanced. 

38-40. In my language classes, the language of 

instruction is the target language. 

a) always. 

b) almost always. 

c) sometimes. 

d) rarely. 

e) never. 

38. Elementary. 

39. Intermediate. 

40. Advanced! 

41-43. In my language classes, I give speaking tests; 

a) weekly. 

b) two or three times a semester. 

c) once a s em ester. 

d) never. 

41. Elementary. 

42. Intermediate. 

43. Advanced. 

44-46. Students in my language classes are required to 
use the language laboratory: 

a) daily. 

b) weekly 

c) not required. 

d) other 

44. Elementary. 

45. Intermediate. 

46. Advanced. 

47-49. When speaking with students outside of class: 

a) I almost always use the target language . 

b) I sometimes use the target language. 

c) I almost always speak English. 

47. Elementary. 

48. Intermediate. 

49. Advanced. 



35. / 

36 . / 

37. / 



38. / 

39 . / 

40. / 



41 . / 

42 . / 

43 . / 



44 . / 

45. / 

46 . / 



47 . / 

48 . / 

49. / 
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50. I and/or my department use oral tests for: 

a) placement. 

b) proficiency (meeting a standard for the 
language requirement, graduation, or 
certification) . 

c) placement and proficiency. 

d) none of the above purposes. 

e) other . 50 / 



51 . My college/university has a language require- 
ment based on: 

a) number of semesters/quarters successfully 
completed. 

b) a proficiency exam. 

c) other criteria . 

d) no requirement. 51 



SeptApn JE^Jin: Background Information 

Circle the letter of the one most appropriate answer. 

52. Sex: 

a) female. 

b) male. 



53. Age: 

a) 22-29. 

b) 30-37. 

c) 38-45. 

d) 46 plus. 



54. Native language. 

a) English. 

b) French. 

c) German. 

d) Russian. 

e) Spanish. , 

f) other . 54. / 

55. Are you a native speaker of the language you 
teach? 

a) yes: language 

b) no. 55. / 

56. My highest academic degree is: 

a) Bachelor's degree. 

b) Master's degree. 

c) Doctoral degree. 

d) other _. 56. / 



52. / 



53. / 
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57. My Bachelor*s degree is in: 

a) Language. 

b) Literature. 

c) Language and Literature. 

d) other . 57.. 

58. My Master's degree is in: 

a) Language. 

b) Literature. 

c) Language and Literature. 

d) other . 58., 

59. My Doctoral degree is in: 

a) Language/ linguistics. 

b) Literature. 

c) Language and Literature. 

d) other . 59.. 

60. Number of years teaching college-level foreign 
language courses (exclusive of graduate- student 
teaching assignments) : 

a) 1-3. 

b) i|-7. 

c) 8 plus. 60.. 

61. Academic rank: 

a ) Instructor/ Lecturer . 

b) Assistant Professor. 

c) Associate Professor. 

d ) Professor . 

e) other » 61.. 

62. Academic Status: 

a) Tenured. 

b) Non-tenured Tenure Track. 

c) Non-Tenure Track. 62.. 

63. Primary foreign language now teaching: 

a) French. 

b) German. 

c) Russian. 

d) Spanish. 63. /. 

64-67. My oral skills have been evaluated in an oral 
interview- type exami nation : 

a) yes. 

b) no . 

64. as an undergraduate. 64. /. 

65. as a graduate student. 65. /, 

66. in applying for a job. 66. /. 

67. other circumstances . 67. /. 



Thank you for completing this questionnaire. 



PIN 



ERLC 



63 



PROJECT OPT 



UNIVERSITY OF NEW HAMPSHIRE 0.p«n,.ntof And.ntandMod.n un,u«...„. ut.r,tur- 

DURHAM, NEW HAMPSHIRE 03824 Co"«9» of ub«r.i Ant 

Murkland Hall 



QOESTIONNAIHE II 



As a participant in PROJECT OPT, you have had the opportunity to learn 
about and practice using the ACTFL/ETS Oral Proficiency Interview. Now we 
would like you to reflect on your experiences and answer the questions in 
this survey regarding use of the process in foreign language courses at the 
college-level . 

In some sections of this questionnaire, you are asked to give specific 
data from the log you kept while completing the second set of 16 tapes. In 
other sections, you are asked to comment on various aspects of the process 
and its application to foreign language courses. Please answer the latter 
questions as fully as you can, using additional sheets if necessary. 
Before you start to complete the questionnaire, please indicate your PIN on 
the line below. This information is necessary for data analysis purposes. 

Please return the completed questionnaire by May 25, 1983 to: 

PROJECT OPT 
AMLL/Murkland 209 
University of New Hampshire 
Durham, NH 03824 

Thank you for your assistance in this phase of the project. We look 
forward to your return to Durham in August. It you have any questions, 
please feel free to contact Sandi Mayewski or me at (603) 862-1218 



Barbara H. Wing 
Project Director 



PIN 



Date_ 



1 May 1983 
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Queation Jj. The questloas In this section relate to time factors. 

1.1 Please supply the following information using data from your log: 

Interview # Course Name Rating Preparation Interview Rating 

Time* Time Time 



A-1 












A-2 












A-3 












A-4 












A-5 












A-6 












A-7 












A-8 






- 






B-1 












B-2 












B-3 












B-4 












B-5 












B-6 












B-7 












B-8 












Total Time 













* Preparation Time: Indicate time spent arranging for interview, reviewing 
techniques, assembling equipment, etc. 
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Question 1 (continued) 

1.2 Do you consider the time spent ou preparing, interviewing and rating 
worthwhile with respect to the following: (Please explain) 

1.2.1 Tour own profesaional development: 



1.2.2 Student development: 



1.2.3 Program development: 



1.2.4 Other: 



1.3 Do you have any suggestions for making more efficient use of time in 
the three phases of preparing, interviewing and rating? Please 
explain. 



\ 



1.4 Does the time factor have implications for use of the oral proficiency 
interview in foreign language classes? Please explain. 
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Question Zl The questions In this section relate to facilities and 
equipment used. 

2.1 Describe the facilities and equipment that you used while conducting 
the Interviews. 



2.2 Was arrangement and/or acquisition of equipment a problem for you? 
Please explain. 



2.3 If the tape recordings were not clear enough to make valid decisions, 
what caused the problem? 



2-4 What changes, if any, would you make in conditions, facilities and 
equipment to obtain clearer recordings? 
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Quegtion 1l The questions in this section relate to the specific course 
from which you selected the 8 students who comprised the 
group. 

3.1 Describe the specific course from which you selected 8 students to be 
interviewed: 

3 • 1 • 1 Name of course : ' 

3.1.2 Prerequisite for course: 

3.1.3 Number of students enrolled in class: 

3.1.4 Text(s): 



3.1.5 Objectives of course: 



3.1.6 Percentage of in-class time devoted to: 

I*istening R eading ^Grammar explanations 

% in target language 

^Speaking ^Writing ^Cultural topics 

% in target language 

^Other (explain) 

3.2 Summarize the ratings of the 8 students from the class by indicating 

the number of students rated at each level on the scale: 

^Novice Low Intermediate Low Advanced 

Novice Mid Intermediate Mid A dvanced plus 

^Novice High Interaediate High ^Superior 

3.3 Indicate the rating and final course grade for each student: 

Rating Final Course Grade 



A-1 




A- 2 




A-3 




A-4 




A-5 




A-6 




A-7 




A-8 





60 




Question 3 (continued): 

3.4 Did you prepare your students for the interview? If so, in what way? 



3.5 With regard to the particular course from which you selected these 8 
students, what effect has the use of the oral proficiency interview 
had on your thinking about: 

3.5.1 Gfoals and objectives 



3 •5.2 Curriculum development 



3.5*3 Instructional materials 



3.5.4 Instructional techniques 



3.5.5 Testing procedures 



3.5.6 Other 



3.6 Do you plan to make any specific changes in this course, based on your 
experiences with the oral proficiency interview? Explain. 
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Question 3 (continued): 

3«7 Do you plan to suggeat/make any changes in other courses or in your 
language program, based on experiences with oral proficiency testing? 
Explain. 



3.8 In your opinion, what specific functions can the oral proficiency 
interview serve in a language course such as the one from which you 
selected your candidates? 



3*9 What functions can the oral proficiency interview serve in your total 
program? 



3.10 How have you presented the oral proficiency interview to your 
colleagues? 



3.11 How have your colleagues reacted to the oral proficiency interview? 



6 



Question 4 The questions in this section relate to your experiences with 
your own students and with the students who have not studied 
with you. 

4.1 In the intei^iew phase: 

4.1,1 What specific differences in your own behavior and feelings were 
you aware of while interviewing your own students as opposed to 
those with whom you were not acquainted? 



4,1.2 What specific differences in the behavior and feelings of the 
students were you aware of while interviewing your own students 
as opposed to those with whom you were not acquainted? 



4.2 In the rating phase, what specific differences in your own behavior 
and feelings were you aware of while rating your own students as 
opposed to those with whom you were not acquainted? 



4.3 Do you think that your intez*views with your own students are as valid 
and as reliable as your interviews with students you did not know 
pr e vi ou s ly ? Expl ai n. 



4.4 Do you think it is possible for an interviewer to conduct and rate 
oral proficiency interviews with his/her own students in a valid and 
reliable manner? Explain. 



4.5 What can an interviewer do to obtain valid and reliable interviews 
with his/her own students? 
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Queation 5± The questlona In this section refer to problems you 
encountered In conducting and rating the Interviews. 

5.1 What problems did you have with regard to: 

5.1.1 Arranging for Interviews 



5.1.2 Conducting the Interview: 
5 . 1 »2 » 1 In the Warm-up 

5.1.2.2 In the Level Check 

5.1 .2.3 In the Probes 
5.1 .2.4 In the Wind-up 

5.1.3 Your role In the process: Factors associated with: 

5.1.3.1 Your proficiency In the target language 

5.1.3.2 Assuming the role of interviewer, not teaclaer 

5.1.3.3 Conducting several interviews in a row 
5.1 .3.4 Other 
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5.1.4 The candidate's role: Factors associated with: 
5.1.4.1 (un) willingness to talk 



5.1.4.2 Fear of failure 



5.1.4.3 Other 



5.1.5 Rating the interviews: Factors associated with: 
5.1.5.1 Ratability of sample 



5.1.5.2 Clarity of descriptors in the ACTFL/ETS Rating Scale 



5.1.5.3 Other 



5.2 Please describe any problems that you have not discussed in the 
previous sections. (Use other side if necessary.) 
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Question £l The questions In this section relate to interviewer training 
and further development of the oral proficiency interview. 

6,1 What further training do you need to become more proficient in using 
the interview and rating techniques? 



6.2 What aspects of the oral proficiency interview process need further 
development., clarification or revision? 



6 .3 What questions do you have with regard to use of the oral proficiency 
interview in fgr^lm lammage Qouraes? 



6.4 What topics and activities would you like to see included in the 
August workshop? 



6.5 Funding for PROJECT OPT officially ends in September 1983. If you 
would like to see the project continue, what direction do you 
recommend for the future? 



End of questionnaire 
Thank you 
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PROJECT OPT 



Questionnaire III 
Interviewer Elicitation and Rating Techniques 



Interviewer^^ Evaluator. 



Date. 



(PIN) (name) 

I. The questions in this section concern the elicitation techniques of 
the interviever. They are divided into sections that correspond to 
the- phases of the interview. Using the key, please circle the number 
that most appropriately and consistently describes the performance of 
the interviewer in the four tapes you evaluated. 

KEY: 5 ALWAYS, 4 ALMOST ALWAYS, 3 USUALLY, 4 SELDOM, 1 NEVER 

A. Warm-up 

t. Puts the candidate at ease. 5 4 3 2 1 

2. Uses an appropriate level. 5 4 3 2 1 

3. Uses an appropriate length. 5 4 3 2 1 

B. Level Check 

4. Moves from warm-up to level check smoothly. 5 4 3 2 1 

5. Makes reasonable estimate of candidate's level. 5 4 3 2 1 

6. Samples language in several content areas. 5 4 3 2 1 

7. Stays on topic long enough to get ratable 

sample 5 4 3 2 1 

C. Probes 

9. Probes at appropriate level (i.e., one level 

above language elicited during level check.) 5 4 3 2 1 

10. Probes sufficiently (at least 3 or 4 times) 

to establish ceiling. 5 4 3 2 1 

11. Develops probes naturally from topics of 

conversation in level check. 5 4 3 2 1 

12. Alternates smoothly between level checks and 

probes . ' 5 4 3 2 1 

D. Mini-checks (situations, ask and tell, etc.) 

13. Uses mini-checks at appropriate point (last 
third)intheinterview. 5 4 3 2 1 

14. Introduces mini-checks smoothly and clearly. 5 4 3 2 1 

15. Uses appropriate mini-checks. 5 4 3 2 1 

E. Wind-down 

16. Winds down after probes and mini-checks. 5 4 3 2 1 

17. Leaves candidate with feeling of accomplish- 

tttent . 54321 
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F. Over-all interviewing techniques 

18. Speaks at a normal rate of speed. 5 4 3 2 

19. Is friendly but objective. 5 4 3 2 

20 . Allows candidate to speak without correction . 5 4 3 2 

21. Uses yes /no questions appropriately (at low- 
levels or as a lead-in to higher level). 5 4 3 2 

22. Accepts pauses without prompting candidate. 5 4 3 2 

23. Allows candidate to speak without interruption. 5 4 3 2 

24. Permits candidate to dc moat of the talking. 5 4 3 2 

25. Takes the candidate to the highest level of 
which he/she is capable. 5 4 3 2 1 

II. In this section, the questions concern your perceptions of 
differences, if any, in interviewing and rating techniques relative to 
the interviewer's previous knowledge of the candidate's proficiency. 
In the four interviews you critiqued for this interviewer, two 
candidates were students who had studied with the interviewer and two 
were people who had not studied with him/her. Please answer the 
questions and explain your answers when asked to do so. 

1. Were you able to tell that some of the candidates had studied 
wit'h the interviewer while others had not? 

Yes No 

2. If you answered 'YES', please answer questions 2. a. through 2.h. 
If you answered 'NO', please go to question 3. 

2. a. Did the interviewer explicitly refer to a shared experience 
(e.g., in class) with the student? 
les No 

2.b. Did the student explicitly refer to a shared experience 
with the interviewer? 

Yes No 

2.C. Did the interviewer indicate by some other means that he/she 
knew the candidate? 

Yes _No 

(If 'YES', please explain.) 



2.d. Did the candidate indicate by some other means that he/she 
knew the interviewer? 

Yes ^ No 

(If 'YES', please explain.) 



2.e. Did the interviewer achieve a ratable sample for those 
students he/she seemed to know? 

Yes No 

(If 'NO', please explain.) 
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2.f. Did the interviewer achieve a ratable sample for those 
students he/she did not seem to know? 

Yes No 

(If *NOS please explain.) 



2.g. Do you agree with the rating of those candidates that the 
interviewer seemed to know? 
les No 



(If *NOS please explain.) 



h. Do you agree with the rating of those candidates that the 
interviewer did not seem to know? 

.Yes No 



(If »NOS please explain.) 



III. In this section you are asked to rate the interviewer's overall 
elicitation and rating techniques. Using the key, circle the most 
appropriate number for each item. 

Key: For a person who has participated in a four-day intensive train- 
ing workshop, has received a written critique on a set of eight 
practice tapes, and is now completing a second set of 16 tapes, 
this interviewer rates: 

3 HIGH (Top Third), 2 AVERAGE (Middle Third), 1 LOW (Bottom 
Third) 

1 . Interviewing techniques 3 2 1 

2. Accuracy of rating 3 2 1 

3. Potential for becoming a proficient interviewer 3 2 1 

4. Potential for becoming a proficient rater 3 2 1 

If you have rated this interviewer HIGH or LOW in any of the 4 items, 
please indicate any specific characteristics or behaviors not men- 
tioned in Part I of this questionnaire that contributed to the rating. 
(Use other side if necessary.) 



6/83 End of Questionnaire III 
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PROJECT OPT 
RATING SHEET FOR PRACTICE INTERVIEWS 
Interviewer Project Identification Number (PIN) Workshop Consultant Name 



Candidate No. First Name Course Name Intervi e wer Ratine Consultant Rating (Candidate No.. Rating. Comments) 



Participant name 
Address 



Telephone 



Preferred evening hours 
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PROJECT OPT 
RATING SHEET FOR PROJECT INTERVIEWS: GROUP A 
Interviewer Project Identification Number (PIN) Workshop Consultant Name 



Candidate No. Fir st Name Course Name Interviewer Ratine Consultant Rating (Candidate No., Rating. Comments) 



A- 1 



A- 2 



A- 3 



A- 4 



A- 5 



A- 6 



A- 7 



A- 8 



Participant name 



Address 

80 



Telephone 



Preferred evening hours 
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PROJECT OPT 



UNIVERSITY OF NEVV HAMPSHIRE 



Otpvtnwti of AncMfit aixd Modern LangutQMand Utaraturit 
Collapa of Ubaral Am 
Murklaixi Had 



DURHAM. NEW HAMPSHIRE 03824 



PROJECT OPT LOG 



As you are doing the Phase II Project Interviews, please use these 
pages to keep a log of the following Information: 

1. Time spent on the projeot (page 1.) Please keep an accurate 
record of the amount of time you spend In the folic Ing 
tasks associated with the project: 



2. Perceptions and problems regarding the Interviews (pages 2 
and 3.) Please note any specific or general Impresslonsy 
questions or problems that occur to you while Interviewing 
and rating. 

Cn May 1 , 1 983 you will receive the second and final questionnaire for 
PROJECT OPT. Tou will need information from the sheets in this log to 
complete the questionnaire. The questionnaire should be returned to 
the PROJECT OPT office by May 15i 1 983. Please bring y ur log to the 
August workshop. 



1 . Arranging for interviews. 

2 . Conducting Interviews . 
3 • Rating interviews. 
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INTERVIEW 



page 1 

PROJECT OPT TIME LOG 
Arrangement s Interview Rating 



Total Time: 



page 2 



PROJECT OPT: Perceptions and Problems - Group A 

As you do the Project Interviews, please describe any perceptions or 
problems that occur during the process. The items may relate 
specifically to a particular interview or be of a general nature. 

INTERVIEW 



A-1 



A-2 



A- 3 



A-4 



A-5 



A-6 



A-7 
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PROJECT OPT: Perceptions and Problems - Group B 

As you do the Project Interviews, please describe any perceptions or 
problems that occur during the process. The items may relate 
specifically to a particular interview or be of a general nature. 

INTERVIEW 



B-1 



B-2 



B-3 



B-4 



B-5 



B-6 



B-7 



B-8 

ERIC 
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PROJECT OPT 



Phase II: Project Tapes 
RatlnK Sheet 

Interviewer Evaluator Date 

(PIN) 

Candidate's Interviewer Evaluator Candidate studied with Interviewer? 
Name Rating Rating (YES / NO / Don't Know) 
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Appendix C: 
Raw Data 

1. Questionnaire I: Participant Profile 

2. Questionnaire II: Participant Project Report 

3. Questionnaire III: Consultant Report of 
Interviewer Elicitation and Rating Techniques 
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PROJECT OPT 

Participant Questionnaire 

Pleaae answer all questions by circling the letter of the one most 
appropriate answer. If . you indicate other ." explain briefly in the 
space provided. Read all the answers in each item before marking the 
most appropriate one. Note that the term '•target language" refers to 
the foreign language you are currently teaching. 

5^0 tlon Qjk±i Preparation Response Frequencies 



1. I first learned/studied the language I am now 
teaching: 

a) informally as a child 7 

b) in school (elementary, junior high or high 10 

school) 

c) in a college or university 5 

d) other. 2 

2. I first learned/studied the language I am now 
teaching because: 

a) I needed it to communicate with the people 6 
around me. 

b) I wanted to speak and under stand the language. 9 

c) It was a requirement for my course of study. 8 

d) other 1 

3. - My undergraduate major field of study was: 

a) French 6 

b) German 3 

c) Spanish 5 

d) Russian 3 

e) other 7 

4. As an undergraduate I: 

a) received academic credit for a study-abroad 

program in a target-language country. 9 

b) did not receive academic credit for a study- 
abroad program, 15 

5. As an undergraduate, I lived or travelled in a 
target language country (exclusive of study for 
credit) for a total of: 

a) more than three months. 

b) less than three months. 8 

c) not at all. ^ 

12 

6. As an undergraduate student: 

a) I had a formal course in foreign language 

teaching. 9 

b) I did not have a formal course in foreign 
language teaching. 
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7. As an undergraduates I: 

a) student- taught a foreign language. 10 

b) did not student • teach a foreign language. 14 

8. As a graduate student: 

a) I received academic credit for study In a 7 
target language country. 

b) I did not receive academic credit for study 17 
In a target language country, 

9. As a graduate student I lived or travelled in a 
target country (exclusive of study for credit) for 



a total of: 

a) three months or more. 13 

b) less than three months. 4 

c) not at all. 7 

10. As a graduate student I: 

a) had a formal course in foreign language 9 
teaching . 

b) did not have a formal course in foreigu 15 
language teaching . 

11. As a graduate student I taught a language course: 

a) regularly. 17 

b) occasionally. 4 

c) once. 1 

d) never. 2 



TVQ: Professional Growth 

12. In the past three years I have spent a total of 
In a target language country. 

a) no time. 

b) 1-5 weeks. 3 

c) 6-15 weeks. 5 

d) 15 weeks - 9 months. 3 

e) more than 9 months. 7 

6 

13. I have served as a group leader for students 
on a trip to a foreign country: 

a) never. 

b) 1-3 times. 1 

c) 3-6 times. ^ 

d) more than 6 times. 

12 

14. I am a member of foreign language 

associationC s ) . 

a) state/ regional. 

b) national/international. 

c) (a) and (b). 3 

d) no. 19 

1 
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15. I attend state and national professional meetings: 



a) two or more times a year. 15 

b) once a year. 6 

c) every 2 or 3 years. 2 

d) every 5 or 6 years. 0 

e) almost never. 1 

16. At professional meetings I attend a 

session on foreign language teaching methodology. 

a) always. 8 

b) usually. 7 

c) occasionally. 7 

d) never. 1 

no response: 1 

17. I read foreign magazines or newspapers In the 
target language: 

a) weekly. I3 

b) monthly. 7 

c) quarterly . i| 

d) never . 0 

18. I read a professional Journal of forel^-n 
language teaching: . 

a) regularly. 11 

b) occasionally. 12 

c) never. 1 

19. Previous to this workshop I have: 

a) attended a short session on oral proficiency ^ 
testing (1-4 hours). 

b) attended a day-long workshop. 2 

c) attended a workshop for more than one day. 2 

d) not attended any such sessions. 16 



Section Three : Curriculum and Instruction 

Questions 20-25 are to be answered for three levels of language 
Instruction (elementary, Intermediate, advanced). Write the letter of 
the appropriate answer in the blank following each question number. 

20-22. In my- department, there are stated oral 
objectives for language courses. 

a) yes. 

b) no. 

20. Elementary level. 

21. Intermediate level. 

22. Advanced level. 



a:12, b:12 
a:12, b:12 
a: 7, b:15 
no response: 2 
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23-25. At my institution students have access to a: 

a) library-type language laboratory, 

b) a dial-access language laboratory* 

c) no language laboratory, 

d) other laboratory facilities. 



23- 
24. 
25. 



Elementary . 
Interdemediat e • 
Advanc ed , 



a:19, b:4, c:0, d:1 
a:19, b:3, c:1, d:1 
a: 14, b:4, c:4, no response; 2 



For the following questions, answer only for those levels that you 
have taught in the past two years. 

26-28. In the absence of departmental objectives, I 
I have developed specific oral objectives for 
my language courses. 

a) yes. 

b) no . 

c) notapplicable. 



26 • 

27. 
28. 



El ement ary . 
Intermediate . 
Advanc ed« 



a:6, b:9, c:7, no response:2 
a:7, b:8, c:6, no response:3 
a: 8, b:6, c;6, no response: 4 



29-31 . In my language classes (exclusive of courses 
specifically designed for conversation), a 
student's speaking ability constitutes 
of the final course grade. 

a) 0-10 % 

b) 11-20 % 
g) 21-30 % 

d) 31-40 % 

e) more than 40 % . 



29. Elementary, 

30. Intermediate. 

31. Advanced, 



a:7» b:4, c:4, d:6, e:2, no response: 1 
a:39.b:6, 0:7* d:2, e:0, no response;6 
a:6, b:3, o:3» d:1, e:1, no response;10 



32-34. The average size of my language classes is: 

a) 1-8 students, 

b) 9-16 students. 

c) 17-24 students . 

d) 25-30 students • 

e) more than 30 students . 

32. Elementary 

33. Intermediate. ^-9, c:7, d:2, e:3, no response:1 

34. Advanced. ^^3, b:9, c:4, d:4, esO, no response:4 

a:8, b:6, c:2, d:1, e:0, no response:7 



4 



35-37. In my opinion, a reasonable class 
develop speaking skills is: 

a) 1-8 students. 

b) 9-16 students, 

c) 17-24 students, 

d) 25-32 students, 

e) more than 32 students. 



size to 



35, 
36. 
37. 



Elementary • 
Intermediate . 
Advanced. 



a:8, b:13> c:2, d:0, e:0, no response:1 
a:6, h:^5f c:0, d:0, e:0, no response:3 
a:10, b:9, c:1y d:Oy e:0, no response:4 



38-40. In my language classes, the language of 

instruction is the target language, 

a) always. 

b) almost always. 

c) sometimes. 

d) rarely. 

e) never. 



38. 
39. _ 
40. 



Elementary . 
Intermediate . 
Advanced'. 



a:6, b:9, c:7, d: 1 , e: 1 

a:7, b:8, c:4, d:1, e:0, no response: 4 

a: 14, b:3, c:1, d:0, e:0, no response:6 



41-43. In my language classes, I give speaking tests: 

a) weekly. 

b) two or three times a semester. 

c) once a semester. 

d) never. 



41 . 
42. 
43. 



Elementary . 
Intermediate . 
Advanced* 



a:6, b:7, c:7, d:3, no response:1 
a:1, b:7, c:7, d:4, no response: 5 
a:0, b:6, c:4, d;5, no response: 9 



44-46 



Students in my language classes are required to 
use the language laboratory: 

a) daily. 

b) weekly 

c) not required. 

d) other , 



44. 
45. 
46. 



Elementary . 
Intermediate . 
Advanced . 



a:7, b:13, c:4, d:0 

a:0, b:l4, c:4, d:1, no response: 5 

a:0, b:6, c:8, d:1, no response:9 



47-49. When speaking with students outside of class: 

a) I almost always use the target language. 

b) I sometimes use the target language. 

c) I almost always speak English. 



47. 
48. 
49. 



Elementary . 
In termedia te . 
Advanced . 



a:0, b:15, c:8, no response: 1 
a:4, b:13, c:3, no response:4 
a:7, b:10, c:0, no response:7 




>2 



50. I 



and/or 
a) 



c) 
d) 
e) 



my department use oral tests for: ' 
placement. 

proficiency (meeting a standard for the 

language requirement, graduation, or 

certification) . 

placement and proficiency. 

none of the above purposes. 

other . 



has a language require- 



successfully 



51 . My college/university 
ment based on: 

a) number of semester s/ quarter s 
compl e ted . 

b) a proficiency exam. 

c) other criteria 

d) no requirement. 



■S.e<?tj-9H Four: Background Information 

Circle the letter of the one most appropriate answer 

52. Sex: 

a) female. 

b) male. 



53 



Age : 
a) 
b). 
c) 
d) 



22-29 . 
30-37. 
38-45. 
46 plus 



54 



Native language. 

a) English. 

b) French. 

c) German. 

d) ' Russian. 

e) Spanish. 

f) other 



55 



a native speaker of the language you 
language_ 



56. 



Are you 
teach? 

a) yes: 

b ) no . 



My highest academic degree 

a) Bachelor's degree. 

b) Master's degree. 

c) Doctoral degree. 

d) other 



is 



2 
2 



4 

14 
0 

no response: 



5 

0 
11 

no response: 1 



11 
13 



2 
9 
5 
8 



17 
3 
3 
0 
1 



7 

17 



0 
9 

14 
1 



6 



57 



58 



59 



61 



My Bachelor's degree is in: 

a) Language. 

b) Literature. 

c) Language and Literature. 

d) other 



My Master's degree is in: 

a) Language. 

b) Literature. 

c) Language and Literature. 

d) other , 

My Doctoral degree is in: 

a) Language/linguistics. 

b) Literature. 

c) Language and Literature. 

d) other . 



60. Number of years teaching college-level foreign 
language courses (exclusive of graduate-student 
teaching assignments): 

a) 1-3. 

b) 4-7. 

c) 8 plus. 



Academic rank: 
a) Instructor/Lecturer 
Assistant Professor 
Associate Professor. 
P ■'Of essor . 
other 



b) 
c) 
d) 
e) 



62. Academic Status: 

a) Tenured. 

b) Non-tenured Tenure Track. 

c) Non-Tenure Track. 

63. Primary foreign language now teaching: 

a) French. 

b) German. 

c) Russian. 

d) Spanish. 

64-67. My oral skills have been evaluated in an oral 
interview- type examination: 

a) yes. 

b) no. 

64. as an undergraduate. 

65. as a graduate student. 

66. in applying for a Job. 

67. other circumstances 



5 
5 
8 
6 



4 
10 

9 
1 



2 
10 

3 
2 

no response: 



10 
6 
8 



8 

7 
4 
4 
1 



10 
5 
9 



8 
4 
4 
8 



a:4, b:20 

a:3, b:21 

a:8, b:l6 

a,:1, b:20 



Thank you for completing this questionnaire 



PIN 
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Questionnaire II: Participart Project Report 
Raw Data 



QUESTION 1; 
1.1: 



Totals: 



Participant 


Preparation 


Interview 


Rating 


ID Number 


Time 


Time 


Time 


1 


47 min. 


364 


min. 


360 min. 


2 


(little or no) 


320 


min. 


320 min. 


ii 


60 


mill. 


382 


min. 


505 min. 


5 


i»2 


min. 


380 


min. 


123 min. 


6 


155 


min. 


332 


min. 


410 min. 


9 


55 


min. 




min. 


120 min. 


10 


320 


min. 


Jifln 
MOU 


min* 


480 min. 


11 


i»02 


min. 


2ii8 


UlXXl « 


645 min. 


12 


510 


min. 


480 


min. 


480 min. 


13 


90 


min. 


266 


min. 


233 min. 


14 


i»50 


min. 


480 


min. 


540 min. 


15 


201 


min. 


338 


min. 


298 min. 


16 


105 


min. 


400 


min. 


55 min. 


18 
19 






420 


min. 


450 min. 


20 






285 


min. 


245 min. 


21 


160 


min. 


275 


min. 


295 min. 


22 












23 


175 


min. 


290 


min. 


170 min. 


24 


245 


min. 


345 


min. 


345 min. 


25 


345 


min. 


380 


min. 


510 min. 


27 


120 


min. 


340 


min. 


300 min. 



1.2: Time Spent Worthwhile 

A. Your own development: Yes 21; No 1; Perhaps 1 

B. Students' development: Yes 17; No 3; Blank 3 

C. Program development: 

less grammar, more conversation 
to be seen (3) 

yes but more time will tell (4) 
yes (12) 
blank (1) 

what's all this data to be used for? 

D. Other: 

blank (20) 

students eager to have a rating (2) 
use video 

1.3: Suggestions for more efficient use of timme 
none (5) 

get a testing room 
better preparation 

initial contact by mail, distribute explanation in written form 
rate right after, skip taping (2) 
need more practice 

have grid in front of you during interviews 
prepare list of questions 
more situations 

less preparation, gut feeling good indication 

use secretary for sign-up 

try not to listen to tapes so much (2) 

check list of things to look for at each level 
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shorter interviews at lower level 
skip preliminaries 

1.4: Time factor implications 

difficult because of size of class 

written final, short 

oral interview during finals, arranged 



QUESTION 2: 

2.1: Notes: put a towel under microphone to reduce random noise 
put a sign on door to prevent interruptions 

2.2: Was equipment au?rangement a problem? 

no (22) 
blank (1) 

2.3: Reason for unclear recordings: 

timidity of interviewees 
background noises (4) 
students mumbling (2) 
dying batteries 
interviewing technique (2) 
video, mike too far away 
table--top mike 
unravelled tape 

did not check one recording level (2) 

cell on mike bad 

mll:e too far away 

bad tape 

blank (8) 

no problems 

forgot to turn recorder on 
2.4: Changes in facilities, etc.: 

better recorder (6) 

lab psychology better than office (2) 

plug instead of batteries 

better quality sound in lab, lab office more relaxed (2) 
place mike nearer 

better surroundings (but did O.K.) (8) 
blank (3) 
none (3) 

would not tape more than one per tape 

QUESTION 3: 

3.4: Did you prepare students? 
yes (20) 

general meeting (1) 
handout (1) 

each student got copy of quidelines (3) 
explained purpose (15) 
no (3) 

mock interviews (1) 
a little 

students did informal estimates of themselves 
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3.5: Effect on course 
A. Goals: 

same (8) 

more speaking (4) 
too muoh time (1) 
none (2) 

made me think about objectives (2) 

impress on students what they are conanunicating and to whom 
clarified 

relate rating to course expectations 
rework the entire course 

chemge objective to adhere more to grammar grid (2) 

Curriculum development: 

proposed two new courses 
use more role playing (4) 
slow change 

build in more different types of language (2) 
blank (4) 

reworking entire sequence 
curriculum already incorporates 
none (7) 

class vs. proficiency awareness 
more audio 

pre- and post-course interviews 

C. Instructional materials: 

same (2) 

choose more OP oriented texts (3) 
none (4) 
blank (3) 

use conversation cards (like situation cards) (2) 
class vs. proficiency awareness 
obtained "intermediate" teaching aids 
minimal (2) 

select better exercises to go with OP 

reworking the entire curriculum 

use more native speaker materials 

break students up into conversation groups 

D. Instructional techniques: 

role playing (2) 

interview technique is not a teaching technique 
more group work 

more free exercises, fewer drills (11)* 
blank (2) 

reworking entire curriculum 
class vs. proficiency awareness (2) 
same (2) 
none (1) 

E. Testing procedures: 

use at end of course 
more oral (7) 
same (5) 
blank (k) 
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role play 

class vs. proficiency awareness 

oral testing on a more regular basis (2) 

always gave orals but reconsidering format 

F. Other: 

blank (19) 

used for placement 

used to evaluate achievement 

start conversation early in the course 

used to teach TA's, great help 

3.6: Specific changes in the course: 

change text and instructional materials (2) 
situation use 
stress oral (4) 
restructure all courses 
no (9) 

only in testing (2) 

more speaking if in power to do so (2) 
blank (2) 

3.7; Changes in other courses: 

stress oral in final grades 

use situations and role playing (2) 

use interview for placement (2) 

use interview for pre- and post-course testing (2) 

use after intermediate (2) 

change dealing with students 

general direction for dept. 

muse more questions 

use test as a waiver for course 

will try to convince colleagues of multi-section courses 
stress oral in all courses (2) 

persuade colleagues to use oral testing in 2nd and 3rd years 

will readjust beginning program to guidelines 

beginning courses use team approach and one personal interview 

interview rating becomes part of permanent record 

will not use, too much to add to already overburdened schedule 

realistic approach; will use test more often 

will use to find out why students who study abroad don*t 

improve much 
no, not this university !!!! 
no, we already are in step 

will use for elementary students after school program 

3.8: Function in course you selected from: 

establish an expected performance level 

correspendence between grid and OPT can help to indicate 

acquisition 
use only to improve discussion 

provides better basis for evaluating in course performance 

pre- and post-course evaluation (6) (not for grading, 1) 

make course more relevant (2) 

reinforce oral therefore motivate students (5) 

external agency therefore motivates students 

placement for freshmen 

evaluate proficiency 
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diagnostic 
addition to CV 

supplemental for foreign travel but too time consuming for 
course 

evaluating foreign experience acquisition 
none except practice speaking (2) 
none since not conversation course 

3.9: Function In total program: 

end of study (6) 
part of record (4) 
placement (8) 

periodic testing (3) to Increase OP, screening proficiency 
feedback for motivation 
use In conversation course 

perhaps for course grade; problem since Interview favors 

overseas experience 
Ideas for testing, but time factor a problem (2) 
compare with other Institutions results 
blank 

use for pre-, post-overseas experience 
proficiency requirement 
no total program 

use to readjust courses for more realistic expectations 
use to strengthen Intermediate level 

3-10: How presented to colleagues: 

presented In a mini- workshop plus oral and written report (8) 

showed situations to colleagues (2) 

interested administration 

we all (2) participated in the project 

showed practice interviews (4) 

enthusiastically then more realistically 

briefly 

informally (7) 

cool but enthusiastically 

offered to present; no result yet 

yes 

3-11: How colleagues reacted: 

with interest (7). 
like born-again Christians 
in favor but apprehensive 
wanted to be tested (2) 
with polite interest (2) 
used a lot of time (2) 
interested but no following 
blank 

threatened, critical 

little interest (2) 

indifference (2) 

interest and no interest (3) 

high school enthusiastic, college not 

QUESTION 4: 

4.1: The interview phase: 
A. Own behavior: 
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not me but others 

I have all students (2) 

more patient with other students (2) 

none (2) 

more relaxed with mine (6) 

had to resist slanting toward known vocab and grammar (4) 

? since Interview more formal 

only Interviewed other students 

presumed too much about students level 

easier with other students 

nice to get to know nqr students better 

B. Students behavior: 

own students nervous (5) 

own students easier 

no difference (8) 

other students more nervous (4) 

own students had covered all topics before 
blank (3) 

4.2: Rating own students vs. others: 

none (6) 
blank (3) 

own slightly easier 

surprised at difference between class and Interview rating 
harder on my weaker students 

no difficulty but surprise at difference between class and 

Interview rating 
O.K. but would prefer not to rate own 
disappointed with own students (2) 
spent more time on own students 
problems with Interview not rating 
before listening I thought they were doing better 
easy to evaluate own students higher 

4.3: Are own Interviews as valid?: 

yes (13) more problems with Interview than rating 
mostly (2) 
I hope so 
blank (2) 

interviews, yes; rating, I'd rather not rate my own 

by staying away from vocab/areas unknown to students rating may 

be too high 
yes and no 

preconceived idea of own students therefore may have geared the 
interview 

yes but elidtation technique may be better with other students 

4.4: Conduct and rate own students valid?: 

yes but own more difficult 
yes (16) 

taping helps 
. guidelines should be clearer 

avoid slanted questions (3) 

takes discipline 

another person helps 
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no, not possible 
blank 

4.5; Hovr to get good interviews with own students: 

get a colleague to sit in (4) 
follow the guidelines (8) 

prepare set of cards (difficult ones) and make sure you use 

them also on your own students 
record and play back 
rate 2 times (2nd, a day later) 
use the same format for all 
try to be objective 
blank (3) 

concentrate on the bulk rather than small points 

avoid topics dealt with in class 

don»t prepare the students ahead of time 

QUESTION 5: 

5.1: Problems: 

A. arranging for interview 

none (8) 

very difficult (8) 

difficult with other students (5) 

ask for more volunteers than needed 

B. warm-up 

trouble maintaining simple 
none (7) 

differentiation between warm-up and level check 

difficult (2) 

too long (3) 

developmental 

blank (3) 

warm-ups occurred before recorder was on (2) 
pulling into a routine 
overcoming nervousness 

C. level check 

finding the level (when fluency differed from grammar and 
vocab ) 

knew students therefore knew level 
started too high 
too long 

falling into a routine 
ability to test level 4 
O.K. (3) 

didn*t always probe enough 
difficult (4) 
blank (4) 
too cautious 

warm-up and level check, same 

D. probes 

problems making it rational 
haphazard 
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difficulty in eliciting structures probed for 
blank (5) 

problem with own students 

problem with lower students 

problem with breakdown of fluent speakers (2) 

too high 

didn't probe enough 
no problem 

when and where to probe 
need variety of probes 
routine 
somme trouble 
too much time 

supported opinion and news get no answer (2) 
limited topics 



E. wind-up 



none (5) 

what to say at the end (7) 

routine 

blank (5) 

getting it on tape (2) 
too long 
time factor 

5.1.3s Your role: 

A. your proficiency 

none (7) 
O.K. (4) 
dislexic 
very important 

not as proficient as should be (5) 
accent interfered 

inclination to adjust to the level of students 
the greatest factor 
no 4+ to check on 

B. Interviewer I not teacher: 

sometimes teacher (7) 
easy (4) 

felt like rebelling against OPT 
blank 

very hard (7) 
progressing (2) 

C. Several interviews in a row: 

problem (3) 

boredom 

fatique (5) 

made easier (3) 

no more than three 

repeated questions (4) 

one at a time (2) - 

O.K. if enough time between to rate it (2) 
no more than two (2) 

D. Other: 
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blank (17) 

explaining to candidates what the interview is for 

low institutional support therefore trouble getting candidates 

time due to illness 

yes and no questions 

enthusiastic but questions his/her own ability 
5.1.4; Candidate's role: 

A. Willingness to talk: 

very willing (6) 
O.K. (3) 

discussion prior to interview helped (2) 
some problems (4) 
blank (2) 
only one (7) 

B. Fear of failure: 

none (7) 
blank 

at the outset but later O.K. (2) 
some associated interview with grade 
some (3) 

problem with unknown students (2) 

taken care of by discussion before interview (5) 

some with lower level students 

some probl^s caused by early bresikdown 

C. Other: 

blank (16) 

students got stuck on topics 

high grade students who place low were upset 

mumbling 

all were eager to get their results 
use of English is a problem 
sodium penathol helps!! 

5.1.5: Rating the interview: 

A. Ratability of sample: 

no problem (2) 

ball park figure, when a problem, relist ened 
problem with one 
some problems (6) 

love interviews therefore no problem 

difficult 

blank (4) 

could have shortened some interviews by rating in the head 
earlier 

questions on cultural and vocab aspects 

keep referring to the guidelines 

three tapes had technical problems 

0-2 not ratable because of equipment 

2+ not ratable because of my language proficiency 

small group to pick from 

B. Clarity of ACTFL descriptions 
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problem with Interned. Mid and Intermed. High (3) 
problatn with low end of acale 
blank (5) 

will save questions for next meeting 
descriptions curriculum bound 
problon with 4+ and 2 (2) 
scale O.K. but need more practice 
need language- specific guidelines 
some O.K., some bad 

needs work, too complicated, too many sheets to consult 
clear and over- specific 
some are ambiguous 
very good 
workshop helped 

too much like FSI, not academic 
Russian needs work 

5.2: Other problems: 

blank (14) 

time problem (3) 

did not get to listen to tapes till much later 

hard to focus in the middle of school day or exam week 

needs too much time (2) 
situations on cards make students nervous (situation seems to 

be the probl^ 
use of English is a problem 
stkudent personalities cause problems 

3-5 ratings need more discussion — i.e., Parisian vs. southern 

French vs. Canadian French 
Russian needs more level due to low mastery 
I*d rather teach than interview 

QUESTION 6: Interviewer training and further development of interview 

6.1: What further training do you need?: 

interpersonal skills of interviewing-- need to be introduced to 
the art of interviewing in English first 

observe live or videotaped interviews by experienced 
interviewers (4) 

more practice and discussion of rating interview and 
techniques (11): 

elicitation techniques (6): 

- lower levels 

- level checks (taken for granted) 

- probes (2) 

- supported opinion 

- superior level 
rating (3): 

- can place in range but not level (2) 

- still need critique 

more leisure time in which to prepare and do them (5) 
training periods (2): a complete course, another workshop 
clearer descriptions of novice high, intermed. low/mid/high, 

and advanced low 
conduct interviews with partner to check accuracy and become 

more aware of different phases of interview (2) 
become a qualified trainer of interviews 
be interviewed 

practice in varying questions 
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more linguistic skill 

dearer sense from OPT as to what service they are performing 
more interviews 

more controlled evaluation of results 

6.2; What aspect of OPT interview needs further development?: 

standards and guidelines (9) 

OPT V3. education (we still have terminal 2+*s) 

OPT vs. grading 

specific sample questions 

wind*-up 

time factor 

formal at workshop vs. informal on video and cassettes 
experience 

guidelines on vacab and culture 
blank (7) 

vocab- specific situation cards 
6.3; Questions on use of OPT in courses 
blank (12) 

how to use it in grading process (2) 
entrance and exit interview 
reliability (more than one person needed) 
too burdensome to use in courses (2) 
role playing can be used 

work needed on implications for curriculum plan 
should it be used as an achievement test 

how can this be done in one section without the rest of the 

faculty participating? 
compare notes with others 

how many times should a student go through it in a college 
career 

6.4; Activities for August workshop; 
blank (4) 

more practice interviews 

each person present and defend their rating 

watch an experienced interviewer 

session on problems 

summary of info, from this questionnaire (2) 

fine tuning elicitation techniques 

examine types and signs of linguistic breakdown 

how we can get more involved and facilitate 

OPT in university programs (2) 

national standardization 

each participant speak on changes in courses 

practical application in class (2) 

what if candidate more proficient than interviewer? 

development of list of writing and reading in OPT 

situations geared to Russian 

grammar vs. level 

courses and OPT—how to incorporate OPT into curriculum 
talk to consultant 
videotaped interviews (2) 

6.5; Future directions for OPT; 

workshop for trainers to turn colleagues into interviewers and 
spread out evaluation duties (3) 
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workshops to train interviewers (3) 
regional workshops 

involvement of secondary school teachers (2) (fall NHATFL) 
development of proficiency testing as intergral part of program 
and essential part of student records (3): 

- oral interview as standard for schools and colleges 
like SAT or GRE 

- ongoing work with regional group of high schools 
and colleges to make oral, listening, reading and 
writing proficiency testing an integral part of 
programs 

- integrate OPT with other areas of language study to 
truly rate proficiency, not just achievement 

- use as basis for language education 
develop techniques for role-playing situations that allow 

options that get away from translation problems 
periodic demonstrations through the NH Regional Center (NEW 

project) 
blank (4) 

study in each of the languages relationship between proficiency 

levels and course levels 
development of specific materials to facilitate instruction 

dealing with various kinds of language tested for 

narration, description, hypothesis, etc. 
broad explorative direction to access foreign language teaching 

in U.S. and work to improve it 
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Questionnaire III 



Comparison of Interviewer/Evaluator Ratings of Candidates (1): 

Agreed/ Di sag reed 



All Languages 


Language 


Agreed: # of cases 


Disagreed: # of cases 


French 


17/61!{ 


11/39$ 


German 


6/3&% 


10/62$ 


Russian 


1/HH% 


9/56$ 


Spanish 


6/19$ 


26/81$ 


Totals 


36/39$ 


56/6 1$ 



Non-ratable: 5 
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Questionnaire III 



Comparison of In t erviewer/ Evalua tor Ratings of Candidates (2); 

Difference:. # of steps 

All Languages 







Difference: 


# of steps: 






Language 


1 


UP 
2 + 


? UP 


1 


DOWN 
2 + 


*DOWN 


French 


H 






6 


1 


64* 


German 


3 


1 




5 


1 


6 0* 


Russian 


1 


2 


38* 


4 


1 


6 2* 


Spanish 


1 1 


2 


59* 


8 


1 


41* 


Totals 


1 9/79% 


5/21* 


47* 


23/85* 


4/15* 


53* 
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Questionnaire III 



Comparison of In terviewer/ Evaluator Ratings of Candidates (3): 
Differences: Major Boundary/ Within Level 



ALL LANGUAGES 


Language 


Major boundary differences 


Within level differences 


French 


5/45? 


6/55? 


German 


7/70? 


3/30? 


Ru sslan 


4/5 0? 


4/5 0? 


Spanish 


9/41? 


13/59? 


Totals 


25/49? 


26/51? 
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Questionnaire III 



Comparison of Evaluator/ Interviewer Agreement in Ratings of 

Own Students/Not Own Students 



ALL LANGUAGES 



Language 


Own 
Agree 


Students : 
Disagree 


Not Own 
Agree 


Students : 
Disagree 


French 

(n=28) 


5/36? 




12/86!t 


2/^n% 


German 
(n=i6) 


1/13!t 


7/87!t 


5/62% 


3/38% 


Russian 
(n=i6) 


3/38!t 


5/62% 


5/62% 


3/38% 


Spanish 
(n=32) 


3/1 9? 


13/8l!t 


3/^9% 


13/81? 


Totals • 
(n=92) 


12/26!t 


3V7^!t 


25/5H 


21/46? 
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Questionnaire III 



Direction of Evaluator/Interviewer Disagreement in Ratings of 

Own Students/ Not Own Students 

ALL LANGUAGES 



Language 


n 


UP 


DOWN 


N/ R 


n 


DP 


DOWN 


N/R 


French 


9 




5/56? 




2 




2/2 2? 




German 


7 


3/43? 


4/57? 




3 


1/33? 


2/67? 




Russian 


5 


2/H0% 


2/H0% 


1/20$ 


3 




3/1 OOJ 




Spanish 


13 


6/46? 


5/39? 


2/15? 


13 


7/54? 


4/31? 


2/15? 


Totals 


34 


15/44? 


16/47? 


1/9? 


21 


8/38? 


11/52? 


2/1 OJ 
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Appendix D: 

Materials Developed for the Project (not bound in Report) 



1. Oral ProflclencY Testing in Colleg6-.Level Foreign 
Language Programs, Published by ACTFL Materials 
Center, 1984. 

2. Oral JroflQlency Testing. Handout for Workshop 
for the Foreign Language Teachers of Nashua, New 
Hampshire. March 25, 1983. 

3. Orgl ProfiQienov Testing la Russian. Handout for 
Workshop for the Northern New England Chapter of 
the American Association of Slavic and East 
European Languages. 

4. Tegt^nK Qr^l ProfioienQv Through Interview , 
Handout for the Workshop in Foreign Teaching, 
Rivier College, 1984. 

5. POPT (Publication ^ Project OEH- Volume I, 
Number 1 (March, 1984) and Number 2 (Fall, 1984). 
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